A Low-Power Streaming Speech Enhancement Accelerator for Edge Devices

Transformer-based speech enhancement models yield impressive results. However, their heterogeneous and complex structure restricts model compression potential, resulting in greater complexity and reduced hardware efficiency. Additionally, these models are not tailored for streaming and low-power app...

Full description

Saved in:
Bibliographic Details
Published inIEEE open journal of circuits and systems Vol. 5; pp. 128 - 140
Main Authors Wu, Ci-Hao, Chang, Tian-Sheuan
Format Journal Article
LanguageEnglish
Published New York IEEE 2024
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects
Co-design
Complexity
Complexity theory
Computational modeling
Hardware
hardware implementation
low power
Power management
Real-time systems
Speech enhancement
Speech processing
Static random access memory
Time-frequency analysis
transformer
Transformers
Online AccessGet full text

Cover

More Information
Summary:Transformer-based speech enhancement models yield impressive results. However, their heterogeneous and complex structure restricts model compression potential, resulting in greater complexity and reduced hardware efficiency. Additionally, these models are not tailored for streaming and low-power applications. Addressing these challenges, this paper proposes a low-power streaming speech enhancement accelerator through model and hardware optimization. The proposed high performance model is optimized for hardware execution with the co-design of model compression and target application, which reduces 93.9% of model size by the proposed domain-aware and streaming-aware pruning techniques. The required latency is further reduced with batch normalization-based transformers. Additionally, we employed softmax-free attention, complemented by an extra batch normalization, facilitating simpler hardware design. The tailored hardware accommodates these diverse computing patterns by breaking them down into element-wise multiplication and accumulation (MAC). This is achieved through a 1-D processing array, utilizing configurable SRAM addressing, thereby minimizing hardware complexities and simplifying zero skipping. Using the TSMC 40nm CMOS process, the final implementation requires merely 207.8K gates and 53.75KB SRAM. It consumes only 8.08 mW for real-time inference at a 62.5MHz frequency.
ISSN:2644-1225
2644-1225
DOI:10.1109/OJCAS.2024.3387849