Deep Convolutional Neural Network with Structured Prediction for Weakly Supervised Audio Event Detection

• Published in: Applied sciences Vol. 9; no. 11; p. 2302
• Main Authors: Choi, Inkyu, Bae, Soo Hyun, Kim, Nam Soo
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01.06.2019
• Subjects: Acoustics; Annotations; Artificial intelligence; audio event detection; conditional random field; Conditional random fields; Convolution; convolutional neural network; Datasets; Deep learning; Dependence; Error reduction; Information processing; International conferences; Labels; Machine learning; Methods; Neural networks; Post-production processing; Propagation; Recurrent neural networks; Signal processing; Smoothness; Sound; structured prediction; weakly supervised learning; United States > US; Finland
• ISSN: 2076-3417; 2076-3417
• DOI: 10.3390/app9112302

Audio event detection (AED) is a task of recognizing the types of audio events in an audio stream and estimating their temporal positions. AED is typically based on fully supervised approaches, requiring strong labels including both the presence and temporal position of each audio event. However, fully supervised datasets are not easily available due to the heavy cost of human annotation. Recently, weakly supervised approaches for AED have been proposed, utilizing large scale datasets with weak labels including only the occurrence of events in recordings. In this work, we introduce a deep convolutional neural network (CNN) model called DSNet based on densely connected convolution networks (DenseNets) and squeeze-and-excitation networks (SENets) for weakly supervised training of AED. DSNet alleviates the vanishing-gradient problem and strengthens feature propagation and models interdependencies between channels. We also propose a structured prediction method for weakly supervised AED. We apply a recurrent neural network (RNN) based framework and a prediction smoothness cost function to consider long-term contextual information with reduced error propagation. In post-processing, conditional random fields (CRFs) are applied to take into account the dependency between segments and delineate the borders of audio events precisely. We evaluated our proposed models on the DCASE 2017 task 4 dataset and obtained state-of-the-art results on both audio tagging and event detection tasks.