Spatial Audio Feature Discovery with Convolutional Neural Networks

• Published in: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) pp. 6797 - 6801
• Main Authors: Thuillier, Etienne, Gamper, Hannes, Tashev, Ivan J.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.04.2018
• Subjects: acoustic feature discovery; Acoustics; Convolution; Convolutional neural networks; Deep Taylor Decomposition; Ear; Feature extraction; HRTF personalization; Solid modeling; Spatial sound; Training; virtual reality
• ISSN: 2379-190X
• DOI: 10.1109/ICASSP.2018.8462315

The advent of mixed reality consumer products brings about a pressing need to develop and improve spatial sound rendering techniques for a broad user base. Despite a large body of prior work, the precise nature and importance of various sound localization cues and how they should be personalized for an individual user to improve localization performance is still an open research problem. Here we propose training a convolutional neural network (CNN) to classify the elevation angle of spatially rendered sounds and employing Layer-wise Relevance Propagation (LRP) on the trained CNN model. LRP provides saliency maps that can be used to identify spectral features used by the network for classification. These maps, in addition to the convolution filters learned by the CNN, are discussed in the context of listening tests reported in the literature. The proposed approach could potentially provide an avenue for future studies on modeling and personalization of head-related transfer functions (HRTFs).