Improving UAV imaging quality by optical sensor fusion: an initial study

• Published in: International journal of remote sensing Vol. 38; no. 17; pp. 4931 - 4953
• Main Authors: Jabari, Shabnam, Fathollahi, Fatemeh, Roshan, Aditya, Zhang, Yun
• Format: Journal Article
• Language: English
• Published: London Taylor & Francis 02.09.2017; Taylor & Francis Ltd
• Subjects: Cameras; Classification; Color; Colour; Filters; High resolution; Image classification; Imaging techniques; Military technology; Pixels; Quality; Quality assessment; Resolution; Sensors; Sharpness; Signal to noise ratio; Spatial resolution; Support vector machines; Unmanned aerial vehicles
• ISSN: 0143-1161; 1366-5901
• DOI: 10.1080/01431161.2017.1328145

The most frequent application of unmanned aerial vehicle (UAVs) is to collect optical colour images from an area of interest. Thus, high spatial resolution colour images with high amount of signal to noise ratio (SNR) are of great importance in UAV applications. Currently, most UAVs use single sensor colour filter array (CFA) cameras for image collection, within which the Bayer-pattern sensors are the most frequently used ones. Due to the limitations of the CFAs, the quality (in terms of spatial resolution, SNR, and sharpness) of UAV colour images is not optimal. In this article, a sensor fusion solution is proposed to improve the quality of UAV imaging. In the proposed solution, a high-resolution colour (HRC) Bayer-pattern sensor is replaced by a dual camera set containing a panchromatic (Pan) sensor, with the same pixel size and a Bayer-pattern colour (or a four-band multi-spectral) sensor with larger pixel size; the resulting images of the dual camera set are then fused. The enlarged pixel size of the colour sensor provides a higher SNR for the cost of lower spatial resolution. However, the accompanied Pan sensor provides single band images with high SNR and high spatial resolution. Fusing the images of the dual camera set generates colour (or MS) images with high spatial resolution, SNR, and sharpness compensating for the major problems of the Bayer-pattern filters. This replacement solution is initially tested in a laboratory experiment. The results of quality assessments show that the SNR is increased by 2-3 times, the sharpness is improved by around 2 times, and the spatial resolution is increased up to the level of the pan images, while the colour errors remained almost as low as the original colour images. In addition, image classification capability of the images is examined using two methods: Support Vector Machine (SVM) and Maximum Likelihood (ML). The results of image classification also confirmed around 20-40% increase in accuracy. Therefore, the proposed sensor fusion can be a good alternative for UAV colour sensors.