A Review of Differentiable Digital Signal Processing for Music & Speech Synthesis

The term "differentiable digital signal processing" describes a family of techniques in which loss function gradients are backpropagated through digital signal processors, facilitating their integration into neural networks. This article surveys the literature on differentiable audio signa...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Hayes, Ben, Shier, Jordie, Fazekas, György, McPherson, Andrew, Saitis, Charalampos
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 29.08.2023
Subjects
Digital signal processing
Music
Neural networks
Optimization
Speech recognition
Online AccessGet full text

Cover

More Information
Summary:The term "differentiable digital signal processing" describes a family of techniques in which loss function gradients are backpropagated through digital signal processors, facilitating their integration into neural networks. This article surveys the literature on differentiable audio signal processing, focusing on its use in music & speech synthesis. We catalogue applications to tasks including music performance rendering, sound matching, and voice transformation, discussing the motivations for and implications of the use of this methodology. This is accompanied by an overview of digital signal processing operations that have been implemented differentiably. Finally, we highlight open challenges, including optimisation pathologies, robustness to real-world conditions, and design trade-offs, and discuss directions for future research.
ISSN:2331-8422