Accurate delineation of individual tree crowns in tropical forests from aerial RGB imagery using Mask R‐CNN

• Published in: Remote sensing in ecology and conservation Vol. 9; no. 5; pp. 641 - 655
• Main Authors: Ball, James G. C., Hickman, Sebastian H. M., Jackson, Tobias D., Koay, Xian Jing, Hirst, James, Jay, William, Archer, Matthew, Aubry‐Kientz, Mélaine, Vincent, Grégoire, Coomes, David A.
• Format: Journal Article
• Language: English
• Published: Oxford John Wiley & Sons, Inc 01.10.2023; Wiley
• Subjects: Aerial photography; Aerial surveys; Agricultural sciences; Aircraft; Algorithms; Artificial neural networks; Canopies; Carbon cycle; Carbon sequestration; Climate change; Computer Science; Computer vision; Convolutional neural networks; Deep learning; Detectron2; Drought; Environmental Engineering; Environmental Sciences; Forest management; forest monitoring; Forests; Image Processing; Imagery; Lidar; Life Sciences; Machine learning; Mask R‐CNN; Monitoring; Mortality; Neural and Evolutionary Computing; Neural networks; Rainforests; Remote sensing; Sensors; Silviculture, forestry; Surveys; tree crown delineation; Trees; Tropical forests; Vegetation; Malaysia; Deep learning; Detectron2; Tree growth; Forest monitoring; Tree crown delineation; Tree mortality; Tree crown segmentation; Tropical forests; Convolutional neural networks; Mask R-CNN
• ISSN: 2056-3485; 2056-3485
• DOI: 10.1002/rse2.332

Tropical forests are a major component of the global carbon cycle and home to two‐thirds of terrestrial species. Upper‐canopy trees store the majority of forest carbon and can be vulnerable to drought events and storms. Monitoring their growth and mortality is essential to understanding forest resilience to climate change, but in the context of forest carbon storage, large trees are underrepresented in traditional field surveys, so estimates are poorly constrained. Aerial photographs provide spectral and textural information to discriminate between tree crowns in diverse, complex tropical canopies, potentially opening the door to landscape monitoring of large trees. Here we describe a new deep convolutional neural network method, Detectree2 , which builds on the Mask R‐CNN computer vision framework to recognize the irregular edges of individual tree crowns from airborne RGB imagery. We trained and evaluated this model with 3797 manually delineated tree crowns at three sites in Malaysian Borneo and one site in French Guiana. As an example application, we combined the delineations with repeat lidar surveys (taken between 3 and 6 years apart) of the four sites to estimate the growth and mortality of upper‐canopy trees. Detectree2 delineated 65 000 upper‐canopy trees across 14 km 2 of aerial images. The skill of the automatic method in delineating unseen test trees was good ( F 1 score = 0.64) and for the tallest category of trees was excellent ( F 1 score = 0.74). As predicted from previous field studies, we found that growth rate declined with tree height and tall trees had higher mortality rates than intermediate‐size trees. Our approach demonstrates that deep learning methods can automatically segment trees in widely accessible RGB imagery. This tool (provided as an open‐source Python package) has many potential applications in forest ecology and conservation, from estimating carbon stocks to monitoring forest phenology and restoration. Python package available to install at https://github.com/PatBall1/Detectree2 .