Continuous sonification enhances adequacy of interactions in peripheral process monitoring

• Published in: International journal of human-computer studies Vol. 95; pp. 54 - 65
• Main Authors: Hildebrandt, Tobias, Hermann, Thomas, Rinderle-Ma, Stefanie
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.11.2016
• Subjects: Dual task; Industrial monitoring; Multi-modal display; Process monitoring; Quantitative study; Sonification; Process monitoring; Sonification; Quantitative study; Industrial monitoring; Dual task; Multi-modal display
• ISSN: 1071-5819; 1095-9300
• DOI: 10.1016/j.ijhcs.2016.06.002

As many users who are charged with process monitoring need to focus mainly on other work while performing monitoring as a secondary task, monitoring systems that purely rely on visual means are often not well suited for this purpose. Sonification, the presentation of data as (non-speech) sound, has proven in several studies that it can help in guiding the user's attention, especially in scenarios where process monitoring is performed in parallel with a different, main task. However, there are several aspects that have not been investigated in this area so far, for example if a continuous soundscape can guide the user's attention better than one that is based on auditory cues. We have developed a system that allows reproducible research to answer such questions. In this system, the participants’ performance both for the main task (simulated by simple arithmetic problems) and for the secondary task (a simulation of a production process) can be measured in a more fine-grained manner than has been the case for existing research in this field. In a within-subject study (n=18), we compared three monitoring conditions – visual only, visual + auditory alerts and a condition combining the visual mode with continuous sonification of process events based on a forest soundscape. Participants showed significantly higher process monitoring performances in the continuous sonification condition, compared to the other two modes. The performance in the main task was at the same time not significantly affected by the continuous sonification. •We present a system for reproducible research in sonification for process monitoring.•We developed an experiment design to analyze effectiveness in monitoring as secondary task.•We compared continuous sonification to visual-only and auditory alerts.•Continuous sonification significantly enhances the adequacy of interactions.•Participants find continuous sonification significantly more helpful and reassuring.