Real-time detection of deep-sea hydrothermal plume based on machine vision and deep learning

• Published in: Frontiers in Marine Science Vol. 10
• Main Authors: Wang, Xun, Cao, Yanpeng, Wu, Shijun, Yang, Canjun
• Format: Journal Article
• Language: English
• Published: Lausanne Frontiers Research Foundation 24.03.2023; Frontiers Media S.A
• Subjects: Accuracy; Artificial intelligence; Autonomous underwater vehicles; Classification; Datasets; Deep layer; Deep learning; Deep sea; Deep water; Detection; Embedded systems; Hydrothermal fields; hydrothermal plume; Hydrothermal springs; Methods; Mineral exploration; Neural networks; object detection; Plumes; real-time; Transfer learning; Underwater exploration; Vision systems; Work platforms
• ISSN: 2296-7745; 2296-7745
• DOI: 10.3389/fmars.2023.1124185

Recent years have witnessed an increase in applications of artificial intelligence (AI) in the detection of oceanic features with the tremendous success of deep learning. Given the unique biological ecosystems and mineral-rich deposits, the exploration of hydrothermal fields is both scientifically and commercially important. To achieve autonomous and intelligent sampling of the hydrothermal plume by using AUV, this paper proposes an innovative method for real-time plume detection based on the YOLOv5n deep learning algorithm designed with a light-weight neural network architecture to meet the requirements of embedded platforms. Ground truth labeler app LabelImg was used to generate the ground truth data from the plume dataset created by ourselves. To accurately and efficiently detect hydrothermal plumes using an embedded system, we improved the original structure of YOLOv5n in two aspects. First, SiLU activation functions in the model were replaced by ReLU activations at shallow layers and Hard-SiLU activations at deep layers to reduce the number of calculations. Second, an attention module termed Coordinate Attention (CA) was integrated into the model to improve its sensitivity to both channel and spatial features. In addition, a transfer learning training method was adopted to further improve the model’s accuracy and generalizability. Finally, we successfully deployed the proposed model in a low-cost embedded device (NVIDIA Jetson TX2 NX) by using the TensorRT inference engine. We then installed the Jetson TX2 NX into a hovering-type AUV as its vision processing unit and conducted a plume detection test in the water tank. The water tank experimental results demonstrated that the proposed method can achieve real-time onboard hydrothermal plume detection.