Transformer Based Unsupervised Pre-Training for Acoustic Representation Learning

• Published in: Proceedings of the ... IEEE International Conference on Acoustics, Speech and Signal Processing (1998) pp. 6933 - 6937
• Main Authors: Zhang, Ruixiong, Wu, Haiwei, Li, Wubo, Jiang, Dongwei, Zou, Wei, Li, Xiangang
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.01.2021
• Subjects: acoustic representation learning; Acoustics; Conferences; Emotion recognition; Event detection; Signal processing; Speech recognition; Training data; Transformer; unsupervised pre-training

Recently, a variety of acoustic tasks and related applications arised. For many acoustic tasks, the labeled data size may be limited. To handle this problem, we propose an unsupervised pre-training method using Transformer based encoder to learn a general and robust high-level representation for all...