Effects of audio-visual interactions on soundscape and landscape perception and their influence on satisfaction with the urban environment

• Published in: Building and environment Vol. 169; p. 106544
• Main Authors: Jeon, Jin Yong, Jo, Hyun In
• Format: Journal Article
• Language: English
• Published: Oxford Elsevier Ltd 01.02.2020; Elsevier BV
• Subjects: Acoustics; Audio data; Audio-visual interaction; Auditory perception; Cognitive ability; Computer applications; Computer simulation; Environmental conditions; Headphones; Helmet mounted displays; Immersive virtual reality; Landscape; Overall satisfaction; Perception; Perceptual evaluation; Semantics; Sensory integration; Sound sources; Soundscape; Technology utilization; Urban areas; Urban environments; Urban planning; Virtual reality; Visual effects; Visual perception; Audio-visual interaction; Overall satisfaction; Landscape; Soundscape; Perceptual evaluation; Virtual reality
• ISSN: 0360-1323; 1873-684X
• DOI: 10.1016/j.buildenv.2019.106544

This study investigated the relationship between overall satisfaction with the urban environment and its soundscape and landscape based on a subjective evaluation of the audio-visual interaction effects within various urban spaces with different environmental conditions. Actual site conditions were simulated using immersive virtual reality technology in which subjects were provided with visual information via a head-mounted display (HMD) and audio information via head-tracking technology using the first-order ambisonics (FOA) of headphone-based three-dimensional auralization. Subjective responses to sound sources and visual elements such as cognitive response, semantic expression, and rated satisfaction were investigated in audio-visual as well as audio- and visual-only environments. It was shown that the availability of visual information affects the auditory perception of a number of human-made and natural sounds and the availability of audio information affects the visual perception of various visual elements. Semantic differential evaluation revealed that pleasantness and eventfulness and overall quality, regularity, spatial impression, and naturalness were the main evaluation factors for soundscapes and landscapes, respectively. One new finding was that audio information affects the perception of the naturalness of a landscape. Audio and visual information had effects of 24 and 76%, respectively, on overall satisfaction. Based on our findings, overall satisfaction models based on audio-visual elements and soundscape and landscape components, which exhibited 31 and 51% explanatory powers, respectively, were proposed. These newly proposed audio-visual interaction-based satisfaction models can be used for better urban planning. •Prediction models considering audio-visual interaction were developed.•Visual information has a 76% impact on urban environment satisfaction.•Audio information has a 24% impact on overall urban environment satisfaction.•The derived models can be used to achieve improved satisfaction in urban planning.