Research and Realization on the key technology of Motion Image Matching Based on UAV

Publications

Share / Export Citation / Email / Print / Text size:

International Journal of Advanced Network, Monitoring and Controls

Xi'an Technological University

Subject: Computer Science , Software Engineering

GET ALERTS

eISSN: 2470-8038

DESCRIPTION

3
Reader(s)
8
Visit(s)
0
Comment(s)
0
Share(s)

SEARCH WITHIN CONTENT

FIND ARTICLE

Volume / Issue / page

Related articles

VOLUME 3 , ISSUE 4 (Oct 2018) > List of articles

Research and Realization on the key technology of Motion Image Matching Based on UAV

Yuri Shebzukhov * / Wang Yubian *

Keywords : UAV, Moving Image, Feature Detection, Image Matching, Image Pre-Processing

Citation Information : International Journal of Advanced Network, Monitoring and Controls. Volume 3, Issue 4, Pages 1-6, DOI: https://doi.org/10.21307/ijanmc-2019-014

License : (CC-BY-NC-ND 4.0)

Published Online: 14-October-2019

ARTICLE

ABSTRACT

With the development of communication network and computer software and hardware technology, especially the emergence of high-precision and high-resolution image sensors, the photography and measurement technology of aerial image have played an increasingly important role in today's geological survey. The traditional aeronautical measurement is carried out by a large manned aircraft. The collected measurement information has a large capacity and a wide shooting range, which is suitable for large-area operations. This type of measurement requires high hardware requirements and is expensive, and it is not suitable for small areas. UAV aerial measurement system is used to measure the small areas, UAV is in small size, with great flight fluctuation, but the data image collected is not accurate enough. In this paper, the common algorithms of image fast matching are compared to conduct in-depth research on gray-level matching and feature-based matching, and SIFT feature matching algorithm based on feature matching is proposed to obtain the measurement image as consistent as possible with the actual scene. The main features of the object to be measured can be obtained through the actual surface area image measurement test, which is of practical significance in the practical low-altitude small area surveying and mapping.

I. INTRODUCTION

UAV is named quad-rotor aircraft or quad-rotor UAV [1], It is a four-propeller sky aircraft with cross-shaped propellers. The UAV can be used to take aerial photos or record video with an optical camera or miniature video recorder. UAV photography measurement system is established on the basis of unmanned aerial vehicle (UAV) mobile platform, it is a kind of advanced measurement technology to achieve high spatial resolution remote sensing image data, it is playing the very major role in the geological landform measurement, disaster reduction, disaster prevention, emergency rescue, emergency treatment, post-disaster reconstruction and so on.

At present, in the aspect of UAV it mainly relies on the image aerial camera to collect the image inland and aboard. The traditional measurement camera is not only expensive, but also needs to carry out film image scanning to obtain the digital image. Its shooting quality is low and the measurement takes a long time. With the development of the UAV aerial survey technology, storage and transmission technology, using the measurement type camera CCD image acquisition has been widely used, the CCD camera has the advantages of a low price, sensors work stability, high sensitivity, the camera CCD cannot direct measurement, the difference of image distortion correction is bigger, so before shot aircraft must be matching the calibration.

II. PREPROCESSING OF UAV IMAGE

As the UAV photography system is equipped with a non-professional measuring digital camera, the performance of the instrument is unstable and the orientation element is uncertain, so bring out the optical distortion error of aerial image. The camera focal length used in this system is fixed, so the distortion difference is the systematic error, which produces the same image for all the collected images.

The inspection of camera can adopt the methods of optical laboratory inspection, laboratory inspection and on-duty inspection. At present, the main test method is laboratory test. The experimental field is composed of some mark points of known space coordinates. In the process of check and correct, the experimental field is photographed by the camera under check, and the internal azimuth elements and other elements affecting the shape of the beam are solved according to the method of single space resection or multi-space resection [2]. In 2D experimental campus, the system uses UAV digital camera Easy Calibrate to check and correct the digital camera SonyRx100), Table 1 shows the detection results and contents, with the origin of coordinates at the lower left corner of the image.

Table I.

THE CALIBRATION RESULTS OF SONY RX100 CAMERA

10.21307_ijanmc-2019-014-tbl1.jpg

Image distortion correction -- indirect method. This method USES the coordinates on the corrected image to calculate the image coordinates of the corresponding points on the original image, and combines the gray interpolation method to realize image correction [3], As shown in figure 1.

Figure 1.

Image distortion correction schematic diagram

10.21307_ijanmc-2019-014-f001.jpg

After years of research, the calculate the deformation The model of error correction is: correction models getting thrown the error correction model),

(1)
Δx=(xx0)(k1r2+k2r4)+p1(r2+2(xx0)2)+2p2(xx0)(yy0)+α(xx0)+β(yy0)10.21307_ijanmc-2019-014-eqn1.jpg
(2)
Δy=(yy0)(k1r2+k2r4)+p1(r2+2(yy0)2)+2p1(xx0)(yy0)10.21307_ijanmc-2019-014-eqn2.jpg

x, y : Image point coordinates x_0 and y_0 with image center as the origin: main point coordinates picture.

(3)
r=(xx0)2+(yy0)210.21307_ijanmc-2019-014-eqn3.jpg

k1, k2, coefficient of radial distortion

p1, p2, coefficient of tangential distortion

α: Non-square scaling factor of pixels

β: The no orthogonal error coefficient of CCD array arrangement.

The space resection method is used to calculate the coordinates of the camera in photography, which improves the precision of high external square elements and the precision of geometric calibration [4].

The precise control of UAV attitude is mainly timely adjusted through the acquired signals of the attitude sensor, which generally includes two types: the angle sensor and the angular velocity sensor, The dip sensor is implemented indirectly by an acceleration sensor from three directions, The output signal values represent the current three axial acceleration values, If the UAV hovers in the air and stays still when the actual geological aerial measurement is made, then obtained acceleration value can be easily converted to obtain the real dip parameter [5]. It is impossible for a drone to remain stationary in the air for a long time in practical applications, When there is wind, the UAV may deviate from a certain direction when it is disturbed. Even if the UAV remains in a horizontal direction, the output value of the acceleration sensor will still deviate from the central value, resulting in the misjudgment value given by the control center. To avoid this kind of situation, often need to introduce three axis in the practical measuring angular velocity sensor and ultrasonic range finder, according to the three axis get up the acceleration and angular velocity and the Z axis direction real-time highly value the rate of change of the acceleration of X, Y axis direction, so as to draw close to actual information of the real Angle [5].

III. UAV IMAGE MATCHING

In most cases, the UAV image mapping method adopts the image matching technology, which recognizes the eponymous point between two images or multiple images through corresponding matching algorithm. The common matching methods mainly include the following two categories: one is based on grayscale matching, the other is based on feature matching [4]. In the actual measurement in this paper, SIFT feature matching algorithm, which is most commonly used in feature matching mode, is adopted for high-precision matching of massive data. SIFT matching algorithm adopts the matching based on local feature values of the image [6]. This algorithm holds invariance for translation, rotation occlusion, etc. Therefore, it has strong stability in actual use. The feature matching process is shown in figure 2.

Figure 2.

Feature matching flow chart

10.21307_ijanmc-2019-014-f002.jpg

A. Pyramid image

Pyramid image refers to the process of decomposing the original image to obtain a series of sub-images with different resolutions. These images are sorted from small to large according to the resolution, forming a group of overlapping pyramid-like images. The matching point is found in the top image, and the matching position is taken as the predicted position of the next layer. The matching result of this layer is taken as the initial matching position of the next layer, and then the matching is conducted successively. The matching result is used as control to match other feature points [7]. This process from top to bottom, from coarse to fine, ensures the reliability of image search process.

In the pyramid image structure, the image is represented by hierarchical structure. At the top of the pyramid structure, data is stored at the lowest resolution possible, As the number of pyramid layers increases, The resolution of the data decreases successively. at the bottom of the pyramid, the data with the highest resolution can be stored to meet the needs of users [8]. In this way, different layers and different resolutions are adopted to store and display according to the needs of users, forming a pyramid structure with a higher resolution to a lower one and a smaller data volume to a larger one. This image pyramid structure is a typical hierarchical data structure used for image coding and progressive image transmission [9]. It is suitable for multi-resolution organization of raster data and image data and is also a lossy compression square of raster data or image data.

B. Image feature point extraction

Feature extraction refers to using a computer to present the image information of the same name in the image, which determines the common features in the image [7]. Image feature extraction generally depends on the distribution of gray in the image, and the position shape and size of features are determined through the information.

SIFT feature matching algorithm consists of two parts [10]. The vector features are extracted from multiple images; SIFT is used to match feature vectors. Scale space representation is an expression based on region. Scale space is defined as the product of Gaussian convolution kernel and remote sensing image. Through the derivation of Koendetink and Babaud, it is proved that Gaussian kernel is the only linear kernel to realize scale transformation) [8].

(4)
{L(x,y,σ)=G(x,y,σ)I(x,y)G(x,y,σ)=12πσ2ex2+y22σ210.21307_ijanmc-2019-014-eqn4.jpg

In formula 4, L(x, y, σ) is the scale space, G(x,y,σ) is the Gaussian convolution kernel, I(x,y) is the remote sensing image, x, y, σ are respectively represented by position parameters and scale parameters. The smaller the scale space factor is, the smaller the scale is.

(5)
D(x,y,σ)=(G(x,y,)G(x,y,σ))*I(x,y)=L(x,y,)L(x,y,σ)10.21307_ijanmc-2019-014-eqn5.jpg

After defining the scale space, the scale space function can be used to build the Gaussian pyramid model, The scale proportion between two adjacent layers and the same rank pyramid affect the scale space function defined between two adjacent layers [11]. The scale between adjacent layers is defined as k. Define the scale factor as σ). D(x,y,σ) is the differential Gaussian pyramid function. At last, each sample point is compared with the adjacent points in the corresponding positions of the upper and lower scale space around the same scale space. If the detection point is local maximum or minimum, it is a candidate point of the image in this scale [12].

IV. THE EXPERIMENTAL TEST

This experiment adopts the UAV model for the average Inspire V2.0 aerial vehicle four axis, the main parameters including the maximum altitude of 4500 meters, maximum rising speed of 5 m/s, the maximum level flight speed of 22 m/s, maximum pitching Angle 35 ° 10 m/s wind power, aircraft cabin image sensor using Sony EXMOR 1/2.3). The image device used in this experiment is Cannon5D Mark II, The image size is 36*24mm. Install Visual Studio 2017 on your laptop and configure Open CV for experimental testing, Using C++ to achieve SIFT image feature point extraction, The matching method is two dimensional feature point Brute Force Matcher, Set a certain threshold to filter the matching results, Using Find Homography function to set RANSAC method to eliminate error matching, SIFT was tested according to the above steps to understand its performance, and the performance was obtained as shown in table 2.

Table II.

FEATURES MATCHING PERFORMANCE TABLE

10.21307_ijanmc-2019-014-tbl2.jpg

The experimental results show that the method has a reasonable matching time. After filtering the threshold value and the basic matrix, the points basically cover the key area of the image. The pixel distribution is uniform with the low error. The system can meet the matching requirements, and the matching test image is shown in figure 3. The accuracy rate of the experimental matching is 91.63%. If the image is improved with light effect, the accuracy can reach 93.25%. Therefore, this research method has good anti-interference and high stability, and can be widely used in low-altitude image matching with interference factors.

Figure 3.

Feature Matching Experiments

10.21307_ijanmc-2019-014-f003.jpg

V. CONCLUSION

Based on the preprocessing theory of aerial survey images, this paper studies the extraction method of image features and USES Visual C++ to realize SIFT extraction of image feature points. Brute Force Matcher, a two-dimensional feature point matching method, was selected for image region matching, Use the Find Homography function to set the RANSAC method to eliminate false matches, In this method, the matching accuracy was 91.63%~93.25% (according to the scene illumination) through the experiment of the geological and geomorphological images of a certain scene, and relatively satisfactory matching effect was obtained. However, due to the deficiencies of UAV itself, it is difficult to compare with professional image processing system. With the further development of communication technology and control technology, UAV will have a breakthrough application and development prospect in low-altitude measurement field in the future.

References


  1. SONG Wenping. Research on Integration of Low Altitude Remote Sensing System of UAV and Related Questions of Image Date Processing[D]. Xi’an: Chang’an University, 2016.06:38.
  2. LI Yifei. Research on PID control in quad rotorcraft[J]. Technology and Market, 2016.07:90.
  3. WANG Jianxiong. Study of the key technology of low altitude phootogrammetry of unmanned aerial vehicle and the practice of large scale topography formation[D]. Xi’an: Chang’an University, 2011.12:54.
  4. YU Huai, YANG Wen. A Fast Feature Extraction and Matching Algorithm for UVA[J].Journal of Electronic &Information Technology, 2016.03:510.
  5. D. G. Lowe. Distinctive Image Features from Scale-Invariant Key points[J]. International Journal of Computer Vision, 2014, 60(2):263.
  6. LI Xiang, WANG Yongjun, LI Zhi. Inter- TriadMisalignment of Vector Field Sensors in Attitude and Heading Reference Systems and Its Calibration[J]. Chinese Journal of Sensors and Actuators, 2017.02:267-270
  7. TAN Xiong, YU Xuchu, LIUJingzheng, HUANG Weijie. Object Fast Tracking Based on Unmanned Aerial Vehicle Video[C]//Proceedings of PACIIA, IEEE Presss, 2010:236.
  8. WANG Donghua, YUE Dawei. Design and Implementation of large remote sensing image correction test system [J]. Computer Programming Skills & Maintenance, 2015.12:118.
  9. Liu Yawei. The review of UAV target detection and tracking method[J]. Aerodynamic Missile Journal, 2016,(9):54-56.
  10. D. G. Lowe. Distinctive Image Features from Scale-Invariant Key points[J]. International Journal of Computer Vision.2014, 60(2).
  11. CUI Zhe. Image’s characteristic point extraction and matching based on SIFT algorithm[D]. Chengdu: University of Electronic Science and Technology of China. 2016.05:21-28.
  12. LU Xiaopan. Empirical Study on the Mapping Precision Based on UAV Low-altitude Photogrammetry[D]. Beijing: China University of Mining and Technology. 2014.06:53-56.
XML PDF Share

FIGURES & TABLES

Figure 1.

Image distortion correction schematic diagram

Full Size   |   Slide (.pptx)

Figure 2.

Feature matching flow chart

Full Size   |   Slide (.pptx)

Figure 3.

Feature Matching Experiments

Full Size   |   Slide (.pptx)

REFERENCES

  1. SONG Wenping. Research on Integration of Low Altitude Remote Sensing System of UAV and Related Questions of Image Date Processing[D]. Xi’an: Chang’an University, 2016.06:38.
  2. LI Yifei. Research on PID control in quad rotorcraft[J]. Technology and Market, 2016.07:90.
  3. WANG Jianxiong. Study of the key technology of low altitude phootogrammetry of unmanned aerial vehicle and the practice of large scale topography formation[D]. Xi’an: Chang’an University, 2011.12:54.
  4. YU Huai, YANG Wen. A Fast Feature Extraction and Matching Algorithm for UVA[J].Journal of Electronic &Information Technology, 2016.03:510.
  5. D. G. Lowe. Distinctive Image Features from Scale-Invariant Key points[J]. International Journal of Computer Vision, 2014, 60(2):263.
  6. LI Xiang, WANG Yongjun, LI Zhi. Inter- TriadMisalignment of Vector Field Sensors in Attitude and Heading Reference Systems and Its Calibration[J]. Chinese Journal of Sensors and Actuators, 2017.02:267-270
  7. TAN Xiong, YU Xuchu, LIUJingzheng, HUANG Weijie. Object Fast Tracking Based on Unmanned Aerial Vehicle Video[C]//Proceedings of PACIIA, IEEE Presss, 2010:236.
  8. WANG Donghua, YUE Dawei. Design and Implementation of large remote sensing image correction test system [J]. Computer Programming Skills & Maintenance, 2015.12:118.
  9. Liu Yawei. The review of UAV target detection and tracking method[J]. Aerodynamic Missile Journal, 2016,(9):54-56.
  10. D. G. Lowe. Distinctive Image Features from Scale-Invariant Key points[J]. International Journal of Computer Vision.2014, 60(2).
  11. CUI Zhe. Image’s characteristic point extraction and matching based on SIFT algorithm[D]. Chengdu: University of Electronic Science and Technology of China. 2016.05:21-28.
  12. LU Xiaopan. Empirical Study on the Mapping Precision Based on UAV Low-altitude Photogrammetry[D]. Beijing: China University of Mining and Technology. 2014.06:53-56.

EXTRA FILES

COMMENTS