A transfer learning and progressive stacking approach to reducing deep model sizes with an application to speech enhancement

• Published in: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) pp. 5575 - 5579
• Main Authors: Sicheng Wang, Kehuang Li, Zhen Huang, Siniscalchi, Sabato Marco, Chin-Hui Lee
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.03.2017
• Subjects: Knowledge engineering; model compression; model stacking; multi-task training; Neural networks; Noise measurement; Speech; Speech enhancement; Stacking; Training; Transfer learning

Leveraging upon transfer learning, we distill the knowledge in a conventional wide and deep neural network (DNN) into a narrower yet deeper model with fewer parameters and comparable system performance for speech enhancement. We present three transfer-learning solutions to accomplish our goal. First...