Modeling short visual events through the BOLD moments video fMRI dataset and metadata

• Published in: Nature communications Vol. 15; no. 1; pp. 6241 - 26
• Main Authors: Lahner, Benjamin, Dwivedi, Kshitij, Iamshchinina, Polina, Graumann, Monika, Lascelles, Alex, Roig, Gemma, Gifford, Alessandro Thomas, Pan, Bowen, Jin, SouYoung, Ratan Murty, N. Apurva, Kay, Kendrick, Oliva, Aude, Cichy, Radoslaw
• Format: Journal Article
• Language: English
• Published: London Nature Publishing Group UK 24.07.2024; Nature Publishing Group; Nature Portfolio
• Subjects: 59; 59/36; 59/57; 631/378/116/2395; 631/378/2613; 631/378/2649/1723; Adult; Artificial neural networks; Brain; Brain - diagnostic imaging; Brain - physiology; Brain mapping; Brain Mapping - methods; Clips; Cortex (parietal); Datasets; Female; Functional magnetic resonance imaging; Human subjects; Humanities and Social Sciences; Humans; Information processing; Magnetic Resonance Imaging - methods; Male; Medical imaging; Metadata; multidisciplinary; Neural networks; Neuroimaging; Parietal Lobe - diagnostic imaging; Parietal Lobe - physiology; Perception; Photic Stimulation; Repositories; Science; Science (multidisciplinary); Temporal lobe; Visual perception; Visual Perception - physiology; Young Adult
• ISSN: 2041-1723; 2041-1723
• DOI: 10.1038/s41467-024-50310-3

Studying the neural basis of human dynamic visual perception requires extensive experimental data to evaluate the large swathes of functionally diverse brain neural networks driven by perceiving visual events. Here, we introduce the BOLD Moments Dataset (BMD), a repository of whole-brain fMRI responses to over 1000 short (3 s) naturalistic video clips of visual events across ten human subjects. We use the videos’ extensive metadata to show how the brain represents word- and sentence-level descriptions of visual events and identify correlates of video memorability scores extending into the parietal cortex. Furthermore, we reveal a match in hierarchical processing between cortical regions of interest and video-computable deep neural networks, and we showcase that BMD successfully captures temporal dynamics of visual events at second resolution. With its rich metadata, BMD offers new perspectives and accelerates research on the human brain basis of visual event perception. There is a need for extensive neuroimaging datasets to facilitate the study of dynamic human visual perception. Here, the authors present a repository of whole-brain fMRI responses to over 1000 short naturalistic video clips across ten human subjects.