Instance segmentation for the fine detection of crop and weed plants by precision agricultural robots

• Published in: Applications in plant sciences Vol. 8; no. 7; pp. e11373 - n/a
• Main Authors: Champ, Julien, Mora‐Fallas, Adan, Goëau, Hervé, Mata‐Montero, Erick, Bonnet, Pierre, Joly, Alexis
• Format: Journal Article
• Language: English
• Published: United States John Wiley & Sons, Inc 01.07.2020; Wiley; John Wiley and Sons Inc
• Subjects: Agricultural production; Agriculture; Algorithms; Application; autonomous robot; Biodiversity; Biodiversity and Ecology; Botanics; convolutional neural network; Corn; Crops; data collection; Deep learning; digital agriculture; Ecology, environment; Ecosystems; electric potential difference; electrodes; Environmental Sciences; Herbicides; Life Sciences; Neural networks; plant detection; Plant sciences; risk; Robots; Segmentation; Semantics; Species; Systematics, Phylogenetics and taxonomy; Vegetal Biology; weed electrification; Weeds; France; United States > US; plant detection; weed electrification; deep learning; digital agriculture; autonomous robot; convolutional neural network; Deep learning; Autonomous robot; Plant detection; Weed electrification; Convolutional neural network; Digital agriculture
• ISSN: 2168-0450; 2168-0450
• DOI: 10.1002/aps3.11373

Premise Weed removal in agriculture is typically achieved using herbicides. The use of autonomous robots to reduce weeds is a promising alternative solution, although their implementation requires the precise detection and identification of crops and weeds to allow an efficient action. Methods We trained and evaluated an instance segmentation convolutional neural network aimed at segmenting and identifying each plant specimen visible in images produced by agricultural robots. The resulting data set comprised field images on which the outlines of 2489 specimens from two crop species and four weed species were manually drawn. We adjusted the hyperparameters of a mask region‐based convolutional neural network (R‐CNN) to this specific task and evaluated the resulting trained model. Results The probability of detection using the model was quite good but varied significantly depending on the species and size of the plants. In practice, between 10% and 60% of weeds could be removed without too high of a risk of confusion with crop plants. Furthermore, we show that the segmentation of each plant enabled the determination of precise action points such as the barycenter of the plant surface. Discussion Instance segmentation opens many possibilities for optimized weed removal actions. Weed electrification, for instance, could benefit from the targeted adjustment of the voltage, frequency, and location of the electrode to the plant. The results of this work will enable the evaluation of this type of weeding approach in the coming months.