Attention-Based Temporal-Frequency Aggregation for Speaker Verification

• Published in: Sensors (Basel, Switzerland) Vol. 22; no. 6; p. 2147
• Main Authors: Wang, Meng, Feng, Dazheng, Su, Tingting, Chen, Mohan
• Format: Journal Article
• Language: English
• Published: Switzerland MDPI AG 10.03.2022; MDPI
• Subjects: Attention; convolutional neural networks; Dictionaries; Discriminant analysis; Frequencies; Frequency domain analysis; Mass media; Neural networks; Neural Networks, Computer; Phonetics; Representations; self-attention; Speaker identification; speaker verification; Standard deviation; temporal-frequency aggregation; Verification; speaker verification; self-attention; convolutional neural networks; temporal-frequency aggregation
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s22062147

Convolutional neural networks (CNNs) have significantly promoted the development of speaker verification (SV) systems because of their powerful deep feature learning capability. In CNN-based SV systems, utterance-level aggregation is an important component, and it compresses the frame-level features generated by the CNN frontend into an utterance-level representation. However, most of the existing aggregation methods aggregate the extracted features across time and cannot capture the speaker-dependent information contained in the frequency domain. To handle this problem, this paper proposes a novel attention-based frequency aggregation method, which focuses on the key frequency bands that provide more information for utterance-level representation. Meanwhile, two more effective temporal-frequency aggregation methods are proposed in combination with the existing temporal aggregation methods. The two proposed methods can capture the speaker-dependent information contained in both the time domain and frequency domain of frame-level features, thus improving the discriminability of speaker embedding. Besides, a powerful CNN-based SV system is developed and evaluated on the TIMIT and Voxceleb datasets. The experimental results indicate that the CNN-based SV system using the temporal-frequency aggregation method achieves a superior equal error rate of 5.96% on Voxceleb compared with the state-of-the-art baseline models.