Real-Time Detection of Hogweed: UAV Platform Empowered by Deep Learning

• Published in: IEEE transactions on computers Vol. 70; no. 8; pp. 1175 - 1188
• Main Authors: Menshchikov, Alexander, Shadrin, Dmitrii, Prutyanov, Viktor, Lopatkin, Daniil, Sosnin, Sergey, Tsykunov, Evgeny, Iakovlev, Evgeny, Somov, Andrey
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.08.2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: aerial imagery; Agriculture; Artificial neural networks; Cameras; Deep learning; edge computing; Embedded systems; Image segmentation; Monitoring; plant phenotype; precision agriculture; Satellites; Segmentation; Task analysis; Unmanned aerial vehicles
• ISSN: 0018-9340; 1557-9956
• DOI: 10.1109/TC.2021.3059819

The Hogweed of Sosnowskyi ( lat. Heracleum sosnówskyi ) is poisonous for humans, dangerous for farming crops, and local ecosystems. This plant is fast-growing and has already spread all over Eurasia: from Germany to the Siberian part of Russia, and its distribution expands year-by-year. In-situ detection of this harmful plant is a tremendous challenge for many countries. Meanwhile, there are no automatic systems for detection and localization of hogweed. In this article, we report on an approach for fast and accurate detection of hogweed. The approach includes the Unmanned Aerial Vehicle (UAV) with an embedded system on board running various Fully Convolutional Neural Networks (FCNN). We propose the optimal architecture of FCNN for the embedded system relying on the trade-off between the detection quality and frame rate. We propose a model that achieves ROC AUC 0.96 in the hogweed segmentation task, which can process 4K frames at 0.46 FPS on NVIDIA Jetson Nano. The developed system can recognize the hogweed on the scale of individual plants and leaves. This system opens up a wide vista for obtaining comprehensive and relevant data about the spreading of harmful plants allowing for the elimination of their expansion.