Lightweight Real-Time Recurrent Models for Speech Enhancement and Automatic Speech Recognition

• Published in: International journal of interactive multimedia and artificial intelligence Vol. 8; no. 6; pp. 74 - 85
• Main Authors: Dhahbi, Sami, Saleem, Nasir, Gunawan, Teddy Surya, Bourouis, Sami, Ali, Imad, Trigui, Aymen, Algarni, Abeer D
• Format: Journal Article
• Language: English
• Published: IMAI Software 01.06.2024; Universidad Internacional de La Rioja (UNIR)
• Subjects: Computational linguistics; Computer industry; Language processing; Natural language interfaces; Neural networks; real-time speech; simple recurrent unit (sru); speech enhancement; speech processing; speech quality; Voice recognition; Saudi Arabia; United States; Pakistan
• ISSN: 1989-1660; 1989-1660
• DOI: 10.9781/ijimai.2024.04.003

Traditional recurrent neural networks (RNNs) encounter difficulty in capturing long-term temporal dependencies. However, lightweight recurrent models for speech enhancement are important to improve noisy speech, while being computationally efficient and able to capture long-term temporal dependencies efficiently. This study proposes a lightweight hourglass-shaped model for speech enhancement (SE) and automatic speech recognition (ASR). Simple recurrent units (SRU) with skip connections are implemented where attention gates are added to the skip connections, highlighting the important features and spectral regions. The model operates without relying on future information that is well-suited for real-time processing. Combined acoustic features and two training objectives are estimated. Experimental evaluations using the short time speech intelligibility (STOI), perceptual evaluation of speech quality (PESQ), and word error rates (WERs) indicate better intelligibility, perceptual quality, and word recognition rates. The composite measures further confirm the performance of residual noise and speech distortion. With the TIMIT database, the proposed model improves the STOI and PESQ by 16.21% and 0.69 (31.1%) whereas with the LibriSpeech database, the model improves STOI by 16.41% and PESQ by 0.71 (32.9%) over the noisy speech. Further, our model outperforms other deep neural networks (DNNs) in seen and unseen conditions. The ASR performance is measured using the Kaldi toolkit and achieves 15.13% WERs in noisy backgrounds. KEYWORDS Real-Time Speech, Simple Recurrent Unit (SRU), Speech Enhancement, Speech Processing, Speech Quality.