Multispectral Semantic Segmentation for UAVs: A Benchmark Dataset and Baseline

• Published in: IEEE transactions on geoscience and remote sensing p. 1
• Main Authors: Li, Qiusheng, Yuan, Hang, Fu, Tianning, Yu, Zhibin, Zheng, Bing, Chen, Shuguo
• Format: Journal Article
• Language: English
• Published: IEEE 09.09.2024
• Subjects: Autonomous aerial vehicles; Cameras; Monitoring; Multispectral images; Oceans; Semantic segmentation; Semantics; Solidago canadensis L; UAV; UAVM dataset; UAVMNet; Vegetation mapping
• ISSN: 0196-2892; 1558-0644
• DOI: 10.1109/TGRS.2024.3457674

Solidago canadensis L. is a typical invasive plant that has become a significant threat worldwide and profoundly impacts local ecosystems. An unmanned aerial vehicle (UAV)-based semantic segmentation system can help in monitoring the spread and location of Solidago canadensis L. To identify the growth range of this species with greater efficiency, we employ a high-speed multispectral camera, which provides richer color information and features with limited resolution, in conjunction with a high-quality RGB camera to construct a segmentation dataset. We construct a validated UAV multispectral (UAVM) dataset comprising 3260 pairs of calibrated RGB and multispectral images. All the images in the dataset underwent semantic annotation at a fine-grained pixel level, with 12 categories being covered. In addition, other plant categories can be employed in precision agriculture and ecological conservation. Moreover, we propose a benchmark model, UAVMNet. With the aid of the feature alignment module and the UAVMFuse module, UAVMNet efficiently integrates multispectral and high-quality RGB image information, enhancing its ability to perform semantic segmentation tasks effectively. To the best of our knowledge, this is the first model to colearn semantic representations via high-quality RGB and paired multispectral information on a UAV platform. We conduct comprehensive experiments on the proposed UAVM dataset.