A High-Rate Extension to Soundstream

In this paper, we propose a high-rate extension of the SoundStream codec which is able to generate almost transparent quality audio at 16 kbps for wideband speech signals. SoundStream shows reasonably good performance at low bit-rates (e.g. around 9 kbps), but its performance does not improve much w...

Full description

Saved in:
Bibliographic Details
Published in2023 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) pp. 1 - 5
Main Authors Kang, Hong-Goo, Skoglund, Jan, Kleijn, W. Bastiaan, Storus, Andrew, Yeh, Hengchin
Format Conference Proceeding
LanguageEnglish
Published IEEE 22.10.2023
Subjects
convolutional transformer
embedding decomposition
Measurement
neural speech coding
Probability density function
Quantization (signal)
Shape
Speech codecs
Speech coding
Transformers
Online AccessGet full text

Cover

More Information
Summary:In this paper, we propose a high-rate extension of the SoundStream codec which is able to generate almost transparent quality audio at 16 kbps for wideband speech signals. SoundStream shows reasonably good performance at low bit-rates (e.g. around 9 kbps), but its performance does not improve much when more bits are used for encoding the latent embeddings. Motivated by experimental results showing that neural audio codec performance is highly related to the characteristics of latent embeddings such as dimensionality, dependency, and probability density function shape, we propose a convolutional transformer architecture and an attention-based multi-scale latent decomposition method that significantly enhances codec performance when quantizing high-dimensional embeddings. Experimental results show the superiority of our proposed model over conventional approaches.
ISSN:1947-1629
DOI:10.1109/WASPAA58266.2023.10248100