Sound-to-Imagination: An Exploratory Study on Cross-Modal Translation Using Diverse Audiovisual Data

• Published in: Applied sciences Vol. 13; no. 19; p. 10833
• Main Authors: Fanzeres, Leonardo A., Nadeu, Climent
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01.10.2023
• Subjects: Acoustics; Analysis; Communications equipment; computational imagination; cross-modal learning; Datasets; deep audiovisual learning; generative adversarial networks (GANs); information bottleneck; Receivers & amplifiers; Semantics; Sound; sound-to-image translation; Supervision; New York; United Kingdom

The motivation of our research is to explore the possibilities of automatic sound-to-image (S2I) translation for enabling a human receiver to visually infer occurrences of sound-related events. We expect the computer to ‘imagine’ scenes from captured sounds, generating original images that depict th...