Optimality and Limitations of Audio-Visual Integration for Cognitive Systems

Multimodal integration is an important process in perceptual decision-making. In humans, this process has often been shown to be statistically optimal, or near optimal: sensory information is combined in a fashion that minimizes the average error in perceptual representation of stimuli. However, som...

Full description

Saved in:
Bibliographic Details
Published inFrontiers in robotics and AI Vol. 7; p. 94
Main Authors Boyce, William Paul, Lindsay, Anthony, Zgonnikov, Arkady, Rañó, Iñaki, Wong-Lin, KongFatt
Format Journal Article
LanguageEnglish
Published Switzerland Frontiers Media S.A 17.07.2020
Subjects
audio-visual illusions
Bayesian integration
cognitive systems
multi-modal processing
multisensory integration
optimality
Robotics and AI
multisensory integration
cognitive systems
audio-visual illusions
multi-modal processing
optimality
Bayesian integration
Online AccessGet full text

Cover

More Information
Summary:Multimodal integration is an important process in perceptual decision-making. In humans, this process has often been shown to be statistically optimal, or near optimal: sensory information is combined in a fashion that minimizes the average error in perceptual representation of stimuli. However, sometimes there are costs that come with the optimization, manifesting as illusory percepts. We review audio-visual facilitations and illusions that are products of multisensory integration, and the computational models that account for these phenomena. In particular, the same optimal computational model can lead to illusory percepts, and we suggest that more studies should be needed to detect and mitigate these illusions, as artifacts in artificial cognitive systems. We provide cautionary considerations when designing artificial cognitive systems with the view of avoiding such artifacts. Finally, we suggest avenues of research toward solutions to potential pitfalls in system design. We conclude that detailed understanding of multisensory integration and the mechanisms behind audio-visual illusions can benefit the design of artificial cognitive systems.
Bibliography:ObjectType-Article-2
SourceType-Scholarly Journals-1
ObjectType-Feature-3
content type line 23
ObjectType-Review-1
Edited by: German I. Parisi, University of Hamburg, Germany
Present address: William Paul Boyce, School of Psychology, University of New South Wales, Sydney, NSW, Australia
This article was submitted to Sensor Fusion and Machine Perception, a section of the journal Frontiers in Robotics and AI
Reviewed by: Christopher C. Berger, California Institute of Technology, United States; Cristiano Cuppini, University of Bologna, Italy
Iñaki Rañó, SDU Biorobotics, Mærsk Mc Kinney Møller Institut, University of Southern Denmark, Odense, Denmark
ISSN:2296-9144
2296-9144
DOI:10.3389/frobt.2020.00094