Road marking extraction in UAV imagery using attentive capsule feature pyramid network

• Published in: International journal of applied earth observation and geoinformation Vol. 107; p. 102677
• Main Authors: Guan, Haiyan, Lei, Xiangda, Yu, Yongtao, Zhao, Haohao, Peng, Daifeng, Marcato Junior, José, Li, Jonathan
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.03.2022; Elsevier
• Subjects: Capsule; Dense atrous convolution; Feature pyramid network; neural networks; neurons; Road markings; spatial data; UAV images; Road markings; Capsule; Dense atrous convolution; Feature pyramid network; UAV images
• ISSN: 1569-8432; 1872-826X
• DOI: 10.1016/j.jag.2022.102677

•The proposed ACapsFPN extracted road markings in UAV images.•A capsule feature pyramid network provided a high-resolution, semantically-strong feature representation.•A novel multi-scale context feature (MCF) descriptor was designed to obtain multi-scale contextual information.•Ternary feature attention modules were designed to improve the accuracy of road marking extraction. Accurately and precisely delineating road-markings from very high spatial resolution unmanned aerial vehicle (UAV) images face many challenges, such as complex scenarios, diverse road marking sizes and shapes, and absent and occluded road markings. To address these issues, we formulate an attentive capsule feature pyramid network (ACapsFPN) by integrating capsule representations with attention mechanisms into the feature pyramid network (FPN), aiming at improving road marking extraction accuracy. Different from the current convolutional neural network (CNN) models based on scalar neuron representations, capsule networks characterize entity features by leveraging vectorial capsule neurons, whose lengths and instantiation parameters contribute to the identification of features and their variants. By constructing a capsule FPN, the ACapsFPN is capable of extracting and integrating multi-level and multi-scale capsule features to provide high-quality and semantically-strong feature abstractions. By formulating a multi-scale context feature descriptor and the ternary feature attention modules, the ACapsFPN can emphasize informative features to generate a class-specific feature representation. Quantitative and qualitative evaluations show the ACapsFPN provides a valuable means for extracting road markings in UAV images under different kinds of complex conditions. In addition, comparative analyses with existing alternatives also demonstrate the superiority and robustness of the ACapsFPN in UAV road marking extraction.