Embodied sound design

• Published in: International journal of human-computer studies Vol. 118; pp. 47 - 59
• Main Authors: Delle Monache, Stefano, Rocchesso, Davide, Bevilacqua, Frédéric, Lemaitre, Guillaume, Baldan, Stefano, Cera, Andrea
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.10.2018; Elsevier
• Subjects: Computer Science; Conceptual design; Human-Computer Interaction; Humanities and Social Sciences; Multimedia; Musicology and performing arts; Sound design tool; Sound synthesis; Conceptual design; Sound synthesis; Sound design tool; Sound synthesis, Conceptual design, Sound design tool
• ISSN: 1071-5819; 1095-9300
• DOI: 10.1016/j.ijhcs.2018.05.007

•Embodied sound design process extensively involves the designer’s vocal apparatus and gestures.•Vocal sketching bridges sound production and perception via motor representation.•We present an automatic system that translates vocal and gestural signals into synthetic sounds.•Voice-driven, perception-based synthesis affords a broad range of concrete to abstract artificial sounds.•The process and tool show how an embodied sound sketching system can support the communication of non-verbal displays in conceptual sound design. Embodied sound design is a process of sound creation that involves the designer’s vocal apparatus and gestures. The possibilities of vocal sketching were investigated by means of an art installation. An artist–designer interpreted several vocal self-portraits and rendered the corresponding synthetic sketches by using physics-based and concatenative sound synthesis. Both synthesis techniques afforded a broad range of artificial sound objects, from concrete to abstract, all derived from natural vocalisations. The vocal-to-synthetic transformation process was then automated in SEeD, a tool allowing to set and play interactively with physics- or corpus-based sound models. The voice-driven process and tool, developed and evaluated through design exercises, show how an embodied sound sketching system can work in supporting the externalisation of sonic concepts.