A method for real-time visual stimulus selection in the study of cortical object perception

• Published in: NeuroImage (Orlando, Fla.) Vol. 133; pp. 529 - 548
• Main Authors: Leeds, Daniel D., Tarr, Michael J.
• Format: Journal Article
• Language: English
• Published: United States Elsevier Inc 01.06.2016; Elsevier Limited
• Subjects: Biofeedback, Psychology - methods; Biofeedback, Psychology - physiology; Brain Mapping - methods; Computational modeling; Computer Systems; Female; Functional magnetic resonance imaging; Humans; Magnetic Resonance Imaging - methods; Male; Medical imaging; Methods; Neurons; Object recognition; Pattern Recognition, Visual - physiology; Photic Stimulation - methods; Real-time signal processing; Real-time stimulus selection; Reproducibility of Results; Sensitivity and Specificity; Studies; User-Computer Interface; Visual Cortex - physiology; Young Adult; Functional magnetic resonance imaging; Real-time stimulus selection; Real-time signal processing; Computational modeling; Object recognition
• ISSN: 1053-8119; 1095-9572; 1095-9572
• DOI: 10.1016/j.neuroimage.2016.02.071

The properties utilized by visual object perception in the mid- and high-level ventral visual pathway are poorly understood. To better establish and explore possible models of these properties, we adopt a data-driven approach in which we repeatedly interrogate neural units using functional Magnetic Resonance Imaging (fMRI) to establish each unit's image selectivity. This approach to imaging necessitates a search through a broad space of stimulus properties using a limited number of samples. To more quickly identify the complex visual features underlying human cortical object perception, we implemented a new functional magnetic resonance imaging protocol in which visual stimuli are selected in real-time based on BOLD responses to recently shown images. Two variations of this protocol were developed, one relying on natural object stimuli and a second based on synthetic object stimuli, both embedded in feature spaces based on the complex visual properties of the objects. During fMRI scanning, we continuously controlled stimulus selection in the context of a real-time search through these image spaces in order to maximize neural responses across pre-determined 1cm3 rain regions. Elsewhere we have reported the patterns of cortical selectivity revealed by this approach (Leeds et al., 2014). In contrast, here our objective is to present more detailed methods and explore the technical and biological factors influencing the behavior of our real-time stimulus search. We observe that: 1) Searches converged more reliably when exploring a more precisely parameterized space of synthetic objects; 2) real-time estimation of cortical responses to stimuli is reasonably consistent; 3) search behavior was acceptably robust to delays in stimulus displays and subject motion effects. Overall, our results indicate that real-time fMRI methods may provide a valuable platform for continuing study of localized neural selectivity, both for visual object representation and beyond. •We develop real-time BOLD signal processing for efficient study of cortical vision.•Adaptive search of stimuli converges on visual selectivities in some searches.•Adaptive search is robust to undesired subject motion and stimulus display errors.•Simpler visual stimuli and search spaces allow more frequent search convergence.•Assumption of single regional selectivity seems flawed, but searches still converge.