What you saw is what you will hear: Two new illusions with audiovisual postdictive effects

• Published in: PloS one Vol. 13; no. 10; p. e0204217
• Main Authors: Stiles, Noelle R B, Li, Monica, Levitan, Carmel A, Kamitani, Yukiyasu, Shimojo, Shinsuke
• Format: Journal Article
• Language: English
• Published: United States Public Library of Science 03.10.2018; Public Library of Science (PLoS)
• Subjects: Acoustics; Analysis; Animal models; Bioengineering; Biology; Biology and Life Sciences; Brain; Computer and Information Sciences; Engineering and Technology; Experiments; Human information processing; Illusions; Information processing; Medicine and Health Sciences; Nervous system; Neurosciences; Perception; Perceptions; Physical Sciences; Relocation; Research and Analysis Methods; Sensory integration; Sensory systems; Social Sciences; Sound; Stimuli; Visual perception; United States > US; Japan; California
• ISSN: 1932-6203; 1932-6203
• DOI: 10.1371/journal.pone.0204217

Neuroscience investigations are most often focused on the prediction of future perception or decisions based on prior brain states or stimulus presentations. However, the brain can also process information retroactively, such that later stimuli impact conscious percepts of the stimuli that have already occurred (called "postdiction"). Postdictive effects have thus far been mostly unimodal (such as apparent motion), and the models for postdiction have accordingly been limited to early sensory regions of one modality. We have discovered two related multimodal illusions in which audition instigates postdictive changes in visual perception. In the first illusion (called the "Illusory Audiovisual Rabbit"), the location of an illusory flash is influenced by an auditory beep-flash pair that follows the perceived illusory flash. In the second illusion (called the "Invisible Audiovisual Rabbit"), a beep-flash pair following a real flash suppresses the perception of the earlier flash. Thus, we showed experimentally that these two effects are influenced significantly by postdiction. The audiovisual rabbit illusions indicate that postdiction can bridge the senses, uncovering a relatively-neglected yet critical type of neural processing underlying perceptual awareness. Furthermore, these two new illusions broaden the Double Flash Illusion, in which a single real flash is doubled by two sounds. Whereas the double flash indicated that audition can create an illusory flash, these rabbit illusions expand audition's influence on vision to the suppression of a real flash and the relocation of an illusory flash. These new additions to auditory-visual interactions indicate a spatio-temporally fine-tuned coupling of the senses to generate perception.