Spectrogram based multi-task audio classification

• Published in: Multimedia tools and applications Vol. 78; no. 3; pp. 3705 - 3722
• Main Authors: Zeng, Yuni, Mao, Hua, Peng, Dezhong, Yi, Zhang
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.02.2019; Springer Nature B.V
• Subjects: Artificial neural networks; Classification; Computer Communication Networks; Computer Science; Convolution; Data Structures and Information Theory; Hierarchies; Model accuracy; Multimedia Information Systems; Neural networks; Pattern recognition; Special Purpose and Application-Based Systems; Convolutional neural networks; Audio classification; Multi-task learning; Deep residual networks
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-017-5539-3

Audio classification is regarded as a great challenge in pattern recognition. Although audio classification tasks are always treated as independent tasks, tasks are essentially related to each other such as speakers’ accent and speakers’ identification. In this paper, we propose a Deep Neural Network (DNN)-based multi-task model that exploits such relationships and deals with multiple audio classification tasks simultaneously. We term our model as the gated Residual Networks (GResNets) model since it integrates Deep Residual Networks (ResNets) with a gate mechanism, which extract better representations between tasks compared with Convolutional Neural Networks (CNNs). Specifically, two multiplied convolutional layers are used to replace two feed-forward convolution layers in the ResNets. We tested our model on multiple audio classification tasks and found that our multi-task model achieves higher accuracy than task-specific models which train the models separately.