A Novel Multi-Feature Fusion Model Based on Pre-Trained Wav2vec 2.0 for Underwater Acoustic Target Recognition

• Published in: Remote sensing (Basel, Switzerland) Vol. 16; no. 13; p. 2442
• Main Authors: Pu, Zijun, Zhang, Qunfei, Xue, Yangtao, Zhu, Peican, Cui, Xiaodong
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01.07.2024
• Subjects: Acoustics; Artificial intelligence; Attention task; Audio data; Comparative analysis; CQT; Data fusion; Datasets; deep learning; Feature extraction; Fourier transforms; Machine learning; Marine environment; Mel-spectrogram; Methods; Modules; multi-feature fusion; Neural networks; Speech; Target recognition; Task complexity; Transfer learning; Underwater; underwater acoustic target recognition; Underwater acoustics; wav2vec 2.0; Wavelet transforms; China
• ISSN: 2072-4292; 2072-4292
• DOI: 10.3390/rs16132442

Although recent data-driven Underwater Acoustic Target Recognition (UATR) methods have played a dominant role in marine acoustics, they suffer from complex ocean environments and rather small datasets. To tackle such challenges, researchers have resorted to transfer learning in an effort to fulfill UATR tasks. However, existing pre-trained models are trained on audio speech data, and are not suitable for underwater acoustic data. Therefore, it is necessary to make further optimization on the basis of these models to make them suitable for the UATR task. Here, we propose a novel UATR framework called Attention Layer Supplement Integration (ALSI), which integrates large pre-trained neural networks with customized attention modules for acoustic. Specifically, the ALSI model consists of two important modules, namely Scale ResNet and Residual Hybrid Attention Fusion (RHAF). First, the Scale ResNet module takes the Constant-Q transform feature as input to obtain relatively important frequency information. Next, RHAF takes the temporal feature extracted by wav2vec 2.0 and the frequency feature extracted by Scale ResNet as input and aims to better integrate the time–frequency features with the temporal feature by using the attention mechanism. The RHAF module can help wav2vec 2.0, which is trained on speech data, to better adapt to underwater acoustic data. Finally, the experiments on the ShipsEar dataset demonstrated that our model can achieve recognition accuracy of 96.39%. In conclusion, extensive experiments confirm the effectiveness of our model on the UATR task.