Neural correlates of audio-visual object recognition: Effects of implicit spatial congruency

• Published in: Human brain mapping Vol. 33; no. 4; pp. 797 - 811
• Main Authors: Plank, Tina, Rosengarth, Katharina, Song, Wookeun, Ellermeier, Wolfgang, Greenlee, Mark W.
• Format: Journal Article
• Language: English
• Published: Hoboken Wiley Subscription Services, Inc., A Wiley Company 01.04.2012; Wiley-Liss; John Wiley & Sons, Inc
• Subjects: Acoustic Stimulation; Adult; Auditory Perception - physiology; Biological and medical sciences; Brain - physiology; Brain Mapping; Female; fMRI; Fundamental and applied biological sciences. Psychology; Humans; Image Interpretation, Computer-Assisted; Investigative techniques, diagnostic techniques (general aspects); Magnetic Resonance Imaging; Male; Medical sciences; multisensory perception; Nervous system; object recognition; Perception; Photic Stimulation; Psychology. Psychoanalysis. Psychiatry; Psychology. Psychophysiology; Radiodiagnosis. Nmr imagery. Nmr spectrometry; spatial congruency; Vision; Visual Perception - physiology; Young Adult; object recognition; fMRI; Nervous system diseases; Radiodiagnosis; Perception; Multisensory integration; Nuclear magnetic resonance imaging; spatial congruency; Recognition; multisensory perception
• ISSN: 1065-9471; 1097-0193; 1097-0193
• DOI: 10.1002/hbm.21254

Multisensory integration assists us to identify objects by providing multiple cues with respect to object category and spatial location. We used a semantic audiovisual object matching task to determine the effect of spatial congruency on response behavior and fMRI brain activation. Fifteen subjects responded in a four‐alternative response paradigm, which visual quadrant contained the object best matched to the sound presented. Realistic sounds based on head‐related transfer functions were presented binaurally with the simulated sound source corresponding to one of the four quadrants. Following a random sequence, the location of the sound corresponded to the quadrant containing the semantically congruent target on half the trials, whereas on other trials the sound arose from an incongruent location. We examined the effects of spatial congruency on response latencies, hit‐rates and fMRI responses. Preliminary behavioral results revealed a significant effect of spatial congruency on response latency or performance for stimuli with noise added. In the fMRI experiment, spatial congruency had a significant effect on the BOLD response. A cluster in the right middle and superior temporal gyrus was more activated when the auditory sound sources were spatially congruent with the semantically matching visual stimulus. In an exploratory post‐hoc analysis, in which we correlated the BOLD signal with the subjects' ability to locate the sound sources, we found a significant cluster in the left inferior frontal cortex, where the BOLD response increased with sound‐source localization performance. Thus spatial congruency appears to enhance the semantic integration of audiovisual object information in these brain regions. Hum Brain Mapp, 2012. © 2011 Wiley Periodicals, Inc.