MAIA-A machine learning assisted image annotation method for environmental monitoring and exploration

• Published in: PloS one Vol. 13; no. 11; p. e0207498
• Main Authors: Zurowietz, Martin, Langenkämper, Daniel, Hosking, Brett, Ruhl, Henry A, Nattkemper, Tim W
• Format: Journal Article
• Language: English
• Published: United States Public Library of Science 16.11.2018; Public Library of Science (PLoS)
• Subjects: Algorithms; Annotations; Artificial intelligence; Artificial neural networks; Autonomous underwater vehicles; Biology and Life Sciences; Cameras; Classification; Collections; Computer and Information Sciences; Computer applications; Data collection; Data Curation - methods; Databases, Factual; Datasets; Digital imaging; Domains; Earth Sciences; Ecology and Environmental Sciences; Engineering and Technology; Environmental monitoring; Environmental Monitoring - methods; Image analysis; Image annotation; Image processing; Image Processing, Computer-Assisted - methods; Inspection; Learning algorithms; Machine Learning; Marine environment; Medical imaging; Methods; Neural networks; Neural Networks (Computer); Ocean floor; Oceanography; Oceans and Seas; Research and Analysis Methods; Training; Underwater vehicles; Vehicles; United Kingdom > UK; Germany
• ISSN: 1932-6203; 1932-6203
• DOI: 10.1371/journal.pone.0207498

Digital imaging has become one of the most important techniques in environmental monitoring and exploration. In the case of the marine environment, mobile platforms such as autonomous underwater vehicles (AUVs) are now equipped with high-resolution cameras to capture huge collections of images from the seabed. However, the timely evaluation of all these images presents a bottleneck problem as tens of thousands or more images can be collected during a single dive. This makes computational support for marine image analysis essential. Computer-aided analysis of environmental images (and marine images in particular) with machine learning algorithms is promising, but challenging and different to other imaging domains because training data and class labels cannot be collected as efficiently and comprehensively as in other areas. In this paper, we present Machine learning Assisted Image Annotation (MAIA), a new image annotation method for environmental monitoring and exploration that overcomes the obstacle of missing training data. The method uses a combination of autoencoder networks and Mask Region-based Convolutional Neural Network (Mask R-CNN), which allows human observers to annotate large image collections much faster than before. We evaluated the method with three marine image datasets featuring different types of background, imaging equipment and object classes. Using MAIA, we were able to annotate objects of interest with an average recall of 84.1% more than twice as fast as compared to "traditional" annotation methods, which are purely based on software-supported direct visual inspection and manual annotation. The speed gain increases proportionally with the size of a dataset. The MAIA approach represents a substantial improvement on the path to greater efficiency in the annotation of large benthic image collections.