Retrospective for the Dynamic Sensorium Competition for predicting large-scale mouse primary visual cortex activity from videos
Understanding how biological visual systems process information is challenging because of the nonlinear relationship between visual input and neuronal responses. Artificial neural networks allow computational neuroscientists to create predictive models that connect biological and machine vision. Mac...
Saved in:
Published in | ArXiv.org |
---|---|
Main Authors | , , , , , , , , , , , , , , , , , , , , , , , , |
Format | Journal Article |
Language | English |
Published |
United States
Cornell University
12.07.2024
|
Online Access | Get full text |
Cover
Loading…
Abstract | Understanding how biological visual systems process information is challenging because of the nonlinear relationship between visual input and neuronal responses. Artificial neural networks allow computational neuroscientists to create predictive models that connect biological and machine vision. Machine learning has benefited tremendously from benchmarks that compare different model on the same task under standardized conditions. However, there was no standardized benchmark to identify state-of-the-art dynamic models of the mouse visual system. To address this gap, we established the SENSORIUM 2023 Benchmark Competition with dynamic input, featuring a new large-scale dataset from the primary visual cortex of ten mice. This dataset includes responses from 78,853 neurons to 2 hours of dynamic stimuli per neuron, together with the behavioral measurements such as running speed, pupil dilation, and eye movements. The competition ranked models in two tracks based on predictive performance for neuronal responses on a held-out test set: one focusing on predicting in-domain natural stimuli and another on out-of-distribution (OOD) stimuli to assess model generalization. As part of the NeurIPS 2023 competition track, we received more than 160 model submissions from 22 teams. Several new architectures for predictive models were proposed, and the winning teams improved the previous state-of-the-art model by 50%. Access to the dataset as well as the benchmarking infrastructure will remain online at www.sensorium-competition.net. |
---|---|
AbstractList | Understanding how biological visual systems process information is
challenging because of the nonlinear relationship between visual input and
neuronal responses. Artificial neural networks allow computational
neuroscientists to create predictive models that connect biological and machine
vision. Machine learning has benefited tremendously from benchmarks that
compare different model on the same task under standardized conditions.
However, there was no standardized benchmark to identify state-of-the-art
dynamic models of the mouse visual system. To address this gap, we established
the Sensorium 2023 Benchmark Competition with dynamic input, featuring a new
large-scale dataset from the primary visual cortex of ten mice. This dataset
includes responses from 78,853 neurons to 2 hours of dynamic stimuli per
neuron, together with the behavioral measurements such as running speed, pupil
dilation, and eye movements. The competition ranked models in two tracks based
on predictive performance for neuronal responses on a held-out test set: one
focusing on predicting in-domain natural stimuli and another on
out-of-distribution (OOD) stimuli to assess model generalization. As part of
the NeurIPS 2023 competition track, we received more than 160 model submissions
from 22 teams. Several new architectures for predictive models were proposed,
and the winning teams improved the previous state-of-the-art model by 50%.
Access to the dataset as well as the benchmarking infrastructure will remain
online at www.sensorium-competition.net. Understanding how biological visual systems process information is challenging because of the nonlinear relationship between visual input and neuronal responses. Artificial neural networks allow computational neuroscientists to create predictive models that connect biological and machine vision. Machine learning has benefited tremendously from benchmarks that compare different model on the same task under standardized conditions. However, there was no standardized benchmark to identify state-of-the-art dynamic models of the mouse visual system. To address this gap, we established the SENSORIUM 2023 Benchmark Competition with dynamic input, featuring a new large-scale dataset from the primary visual cortex of ten mice. This dataset includes responses from 78,853 neurons to 2 hours of dynamic stimuli per neuron, together with the behavioral measurements such as running speed, pupil dilation, and eye movements. The competition ranked models in two tracks based on predictive performance for neuronal responses on a held-out test set: one focusing on predicting in-domain natural stimuli and another on out-of-distribution (OOD) stimuli to assess model generalization. As part of the NeurIPS 2023 competition track, we received more than 160 model submissions from 22 teams. Several new architectures for predictive models were proposed, and the winning teams improved the previous state-of-the-art model by 50%. Access to the dataset as well as the benchmarking infrastructure will remain online at www.sensorium-competition.net. Understanding how biological visual systems process information is challenging because of the nonlinear relationship between visual input and neuronal responses. Artificial neural networks allow computational neuroscientists to create predictive models that connect biological and machine vision. Machine learning has benefited tremendously from benchmarks that compare different model on the same task under standardized conditions. However, there was no standardized benchmark to identify state-of-the-art dynamic models of the mouse visual system. To address this gap, we established the SENSORIUM 2023 Benchmark Competition with dynamic input, featuring a new large-scale dataset from the primary visual cortex of ten mice. This dataset includes responses from 78,853 neurons to 2 hours of dynamic stimuli per neuron, together with the behavioral measurements such as running speed, pupil dilation, and eye movements. The competition ranked models in two tracks based on predictive performance for neuronal responses on a held-out test set: one focusing on predicting in-domain natural stimuli and another on out-of-distribution (OOD) stimuli to assess model generalization. As part of the NeurIPS 2023 competition track, we received more than 160 model submissions from 22 teams. Several new architectures for predictive models were proposed, and the winning teams improved the previous state-of-the-art model by 50%. Access to the dataset as well as the benchmarking infrastructure will remain online at www.sensorium-competition.net.Understanding how biological visual systems process information is challenging because of the nonlinear relationship between visual input and neuronal responses. Artificial neural networks allow computational neuroscientists to create predictive models that connect biological and machine vision. Machine learning has benefited tremendously from benchmarks that compare different model on the same task under standardized conditions. However, there was no standardized benchmark to identify state-of-the-art dynamic models of the mouse visual system. To address this gap, we established the SENSORIUM 2023 Benchmark Competition with dynamic input, featuring a new large-scale dataset from the primary visual cortex of ten mice. This dataset includes responses from 78,853 neurons to 2 hours of dynamic stimuli per neuron, together with the behavioral measurements such as running speed, pupil dilation, and eye movements. The competition ranked models in two tracks based on predictive performance for neuronal responses on a held-out test set: one focusing on predicting in-domain natural stimuli and another on out-of-distribution (OOD) stimuli to assess model generalization. As part of the NeurIPS 2023 competition track, we received more than 160 model submissions from 22 teams. Several new architectures for predictive models were proposed, and the winning teams improved the previous state-of-the-art model by 50%. Access to the dataset as well as the benchmarking infrastructure will remain online at www.sensorium-competition.net. |
Author | Bashiri, Mohammad Wang, Eric Baikulov, Ruslan Wulf, Wolf De Huang, Tiejun Ma, Lei Sinz, Fabian H Onken, Arno Fahey, Paul G Yu, Shan Li, Bryan M Tolias, Andreas S Turishcheva, Polina Froebe, Rachel Qiu, Yongrong Hansel, Laura Hennig, Matthias H Vystrčilová, Michaela Ponder, Kayla Willeke, Konstantin F Ecker, Alexander S Kudryashova, Nina Zhu, Yu Ding, Zhiwei Rochefort, Nathalie L |
Author_xml | – sequence: 1 givenname: Polina surname: Turishcheva fullname: Turishcheva, Polina organization: Institute of Computer Science and Campus Institute Data Science, University of Göttingen, Germany – sequence: 2 givenname: Paul G surname: Fahey fullname: Fahey, Paul G organization: Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, US – sequence: 3 givenname: Michaela surname: Vystrčilová fullname: Vystrčilová, Michaela organization: Institute of Computer Science and Campus Institute Data Science, University of Göttingen, Germany – sequence: 4 givenname: Laura surname: Hansel fullname: Hansel, Laura organization: Institute of Computer Science and Campus Institute Data Science, University of Göttingen, Germany – sequence: 5 givenname: Rachel surname: Froebe fullname: Froebe, Rachel organization: Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, US – sequence: 6 givenname: Kayla surname: Ponder fullname: Ponder, Kayla organization: Department of Neuroscience & Center for Neuroscience and Artificial Intelligence, Baylor College of Medicine, Houston, Texas, USA – sequence: 7 givenname: Yongrong surname: Qiu fullname: Qiu, Yongrong organization: Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, US – sequence: 8 givenname: Konstantin F surname: Willeke fullname: Willeke, Konstantin F organization: Institute for Bioinformatics and Medical Informatics, Tübingen University, Germany – sequence: 9 givenname: Mohammad surname: Bashiri fullname: Bashiri, Mohammad organization: Institute for Bioinformatics and Medical Informatics, Tübingen University, Germany – sequence: 10 givenname: Ruslan surname: Baikulov fullname: Baikulov, Ruslan organization: lRomul, Russia – sequence: 11 givenname: Yu surname: Zhu fullname: Zhu, Yu organization: Beijing Academy of Artificial Intelligence, China – sequence: 12 givenname: Lei surname: Ma fullname: Ma, Lei organization: Beijing Academy of Artificial Intelligence, China – sequence: 13 givenname: Shan surname: Yu fullname: Yu, Shan organization: Institute of Automation, Chinese Academy of Sciences, China – sequence: 14 givenname: Tiejun surname: Huang fullname: Huang, Tiejun organization: Beijing Academy of Artificial Intelligence, China – sequence: 15 givenname: Bryan M surname: Li fullname: Li, Bryan M organization: School of Informatics, University of Edinburgh, UK – sequence: 16 givenname: Wolf De surname: Wulf fullname: Wulf, Wolf De organization: School of Informatics, University of Edinburgh, UK – sequence: 17 givenname: Nina surname: Kudryashova fullname: Kudryashova, Nina organization: School of Informatics, University of Edinburgh, UK – sequence: 18 givenname: Matthias H surname: Hennig fullname: Hennig, Matthias H organization: School of Informatics, University of Edinburgh, UK – sequence: 19 givenname: Nathalie L surname: Rochefort fullname: Rochefort, Nathalie L organization: Simons Initiative for the Developing Brain, University of Edinburgh, UK – sequence: 20 givenname: Arno surname: Onken fullname: Onken, Arno organization: School of Informatics, University of Edinburgh, UK – sequence: 21 givenname: Eric surname: Wang fullname: Wang, Eric organization: Department of Neuroscience & Center for Neuroscience and Artificial Intelligence, Baylor College of Medicine, Houston, Texas, USA – sequence: 22 givenname: Zhiwei surname: Ding fullname: Ding, Zhiwei organization: Department of Neuroscience & Center for Neuroscience and Artificial Intelligence, Baylor College of Medicine, Houston, Texas, USA – sequence: 23 givenname: Andreas S surname: Tolias fullname: Tolias, Andreas S organization: Department of Electrical Engineering, Stanford University, Stanford, CA, US – sequence: 24 givenname: Fabian H surname: Sinz fullname: Sinz, Fabian H organization: Institute for Bioinformatics and Medical Informatics, Tübingen University, Germany – sequence: 25 givenname: Alexander S surname: Ecker fullname: Ecker, Alexander S organization: Max Planck Institute for Dynamics and Self-Organization, Göttingen, Germany |
BackLink | https://www.ncbi.nlm.nih.gov/pubmed/39040641$$D View this record in MEDLINE/PubMed |
BookMark | eNpVkNtKxDAQhoOsuLruK0guvSk0h6btlch6hAXBw3VJ28lupE1qki7ula9u1FVWGJiBb-b_Z-YETYw1cICOKWMkKTilk716iubev6ZpSkVOs4wdoSkrU54KTo7RxyMEZ_0ATdAbwMo6HNaAr7ZG9rrBT2C8dXrs8cL2AwQdtDXfXYODVschs8KddCtIfCM7wL0dPUSoe-m2eKP9KDvcWBfgHcsvDx22WDnbR9aC9afoUMnOw3yXZ-jl5vp5cZcsH27vF5fLZCCElgkRlDAuZKuAZKWoCVNlo1QjOGSExgBomVCSkxxUVkquirYBWTNKeJ3ynM3QxY_uMNY9RGaCk121W7SyUlf_idHramU3VbQXpMzLqHC-U3D2bQQfql77BrpOGohHVywtmCjSjPHYerZv9ufy-3b2CcvUhnU |
ContentType | Journal Article |
DBID | NPM 7X8 5PM |
DatabaseName | PubMed MEDLINE - Academic PubMed Central (Full Participant titles) |
DatabaseTitle | PubMed MEDLINE - Academic |
DatabaseTitleList | PubMed MEDLINE - Academic |
Database_xml | – sequence: 1 dbid: NPM name: PubMed url: https://proxy.k.utb.cz/login?url=http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database |
DeliveryMethod | fulltext_linktorsrc |
Discipline | Physics |
EISSN | 2331-8422 |
ExternalDocumentID | 39040641 |
Genre | Journal Article Preprint |
GrantInformation_xml | – fundername: NIMH NIH HHS grantid: RF1 MH126883 – fundername: NIMH NIH HHS grantid: RF1 MH130416 – fundername: NINDS NIH HHS grantid: U01 NS113294 – fundername: NEI NIH HHS grantid: P30 EY002520 – fundername: NIMH NIH HHS grantid: U19 MH114830 – fundername: NEI NIH HHS grantid: R01 EY026927 |
GroupedDBID | ABJCF AFKRA ALMA_UNASSIGNED_HOLDINGS BENPR BGLVJ FRJ HCIFZ M7S M~E NPM PIMPY PTHSS 7X8 5PM CCPQU |
ID | FETCH-LOGICAL-p1129-1621346adfe1596b13f9cffc64e512512eed36fa417ef59a4f8dceab3214b0473 |
ISSN | 2331-8422 |
IngestDate | Wed Jul 24 06:43:18 EDT 2024 Sat Oct 05 16:37:35 EDT 2024 Wed Oct 16 00:09:21 EDT 2024 |
IsOpenAccess | true |
IsPeerReviewed | false |
IsScholarly | false |
Language | English |
License | This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which allows reusers to copy and distribute the material in any medium or format in unadapted form only, for noncommercial purposes only, and only so long as attribution is given to the creator. |
LinkModel | OpenURL |
MergedId | FETCHMERGED-LOGICAL-p1129-1621346adfe1596b13f9cffc64e512512eed36fa417ef59a4f8dceab3214b0473 |
Notes | ObjectType-Working Paper/Pre-Print-3 ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23 |
OpenAccessLink | https://pubmed.ncbi.nlm.nih.gov/PMC11261979 |
PMID | 39040641 |
PQID | 3083680534 |
PQPubID | 23479 |
ParticipantIDs | pubmedcentral_primary_oai_pubmedcentral_nih_gov_11261979 proquest_miscellaneous_3083680534 pubmed_primary_39040641 |
PublicationCentury | 2000 |
PublicationDate | 2024-Jul-12 20240712 |
PublicationDateYYYYMMDD | 2024-07-12 |
PublicationDate_xml | – month: 07 year: 2024 text: 2024-Jul-12 day: 12 |
PublicationDecade | 2020 |
PublicationPlace | United States |
PublicationPlace_xml | – name: United States |
PublicationTitle | ArXiv.org |
PublicationTitleAlternate | ArXiv |
PublicationYear | 2024 |
Publisher | Cornell University |
Publisher_xml | – name: Cornell University |
SSID | ssj0002672553 |
Score | 1.93185 |
Snippet | Understanding how biological visual systems process information is challenging because of the nonlinear relationship between visual input and neuronal... Understanding how biological visual systems process information is challenging because of the nonlinear relationship between visual input and neuronal... |
SourceID | pubmedcentral proquest pubmed |
SourceType | Open Access Repository Aggregation Database Index Database |
Title | Retrospective for the Dynamic Sensorium Competition for predicting large-scale mouse primary visual cortex activity from videos |
URI | https://www.ncbi.nlm.nih.gov/pubmed/39040641 https://www.proquest.com/docview/3083680534/abstract/ https://pubmed.ncbi.nlm.nih.gov/PMC11261979 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnZ3da9RAEMAXWxF8Eb97VcsKvknEZPc2yWO1lSK0lNLKvR27uV0aaJMjyR3WB_3XnZnN15UK6ksIyRLI_maH2dn5YOyd0Atnjf4YGJCGQILJDHpQgiynkUaXgzALdA0cn6ijC_l1Np0NXfoou6QxH7Ifd-aV_A9VeAZcMUv2H8j2H4UHcA984QqE4fpXjM9sU5VdrmQfMHjgm8yDGijqsspX17Tora9KRKOWFR7PUMDzFUaCBzWQsu_RC4B5U77-xDqvV1Q8pGrsd6q5QW0mKB8Fk_fKemzY7lezfI0xEUPABvauB5FYk3l6Ss2BemnRl95VjoGJQ3uvbzd1U5F5e5BflWs6xA9Hwf160JeYSN_ldeux6yKS6BNtg6YtqbhIiDBIpM9MHgFbXhMxkYKKUb4w1q2q2N2rLbYlQgzlPP45-NYiFcNOCVskdcPu2j_cDoMd2RXnj9mjdkPA9z3dJ-yeLZ6yBxSYm9XP2K8NxhzocWDMW8a8Z8xHjGnUwJiPGHNizFvG3DPmnjHvGHNkzD3j5-ziy-H556OgbZoRLNF0DkKFNfoULkGwVJUJhUsz5zIl7ZSMWTCKhHJahrF101RLl8AEaIMNqwysUvGCbRdlYXcYd8JlkXExNlEFOztOlHTpYpGFNhZSCD1hb7tJnYNSwpMmXVj4jbnAmucJ6Hc5YS_9JM_bP5t3SCYs2Zj-fgAWPN98U-SXVPgc093CNE53__jRV-zhIGiv2XZTrewbsBobs8fufzo8OT3bI1H5DZoHe2g |
link.rule.ids | 230,315,786,790,891,33778 |
linkProvider | ISSN International Centre |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Retrospective+for+the+Dynamic+Sensorium+Competition+for+predicting+large-scale+mouse+primary+visual+cortex+activity+from+videos&rft.jtitle=ArXiv.org&rft.au=Turishcheva%2C+Polina&rft.au=Fahey%2C+Paul+G&rft.au=Vystr%C4%8Dilov%C3%A1%2C+Michaela&rft.au=Hansel%2C+Laura&rft.date=2024-07-12&rft.eissn=2331-8422&rft_id=info%3Apmid%2F39040641&rft.externalDocID=39040641 |
thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2331-8422&client=summon |
thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2331-8422&client=summon |
thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2331-8422&client=summon |