Crowdsourcing seizure detection: algorithm development and validation on human implanted device recordings

• Published in: Brain (London, England : 1878) Vol. 140; no. 6; pp. 1680 - 1691
• Main Authors: Baldassano, Steven N., Brinkmann, Benjamin H., Ung, Hoameng, Blevins, Tyler, Conrad, Erin C., Leyde, Kent, Cook, Mark J., Khambhati, Ankit N., Wagenaar, Joost B., Worrell, Gregory A., Litt, Brian
• Format: Journal Article
• Language: English
• Published: England Oxford University Press 01.06.2017
• Subjects: Adult; Algorithms; Animals; Crowdsourcing - methods; Crowdsourcing - standards; Disease Models, Animal; Electrocorticography - methods; Electrocorticography - standards; Equipment Design - methods; Equipment Design - standards; Humans; Original; Prostheses and Implants; Reproducibility of Results; Seizures - diagnosis; epilepsy; crowdsourcing; experimental models; seizure detection; intracranial EEG
• ISSN: 0006-8950; 1460-2156; 1460-2156
• DOI: 10.1093/brain/awx098

There exist significant clinical and basic research needs for accurate, automated seizure detection algorithms. These algorithms have translational potential in responsive neurostimulation devices and in automatic parsing of continuous intracranial electroencephalography data. An important barrier to developing accurate, validated algorithms for seizure detection is limited access to high-quality, expertly annotated seizure data from prolonged recordings. To overcome this, we hosted a kaggle.com competition to crowdsource the development of seizure detection algorithms using intracranial electroencephalography from canines and humans with epilepsy. The top three performing algorithms from the contest were then validated on out-of-sample patient data including standard clinical data and continuous ambulatory human data obtained over several years using the implantable NeuroVista seizure advisory system. Two hundred teams of data scientists from all over the world participated in the kaggle.com competition. The top performing teams submitted highly accurate algorithms with consistent performance in the out-of-sample validation study. The performance of these seizure detection algorithms, achieved using freely available code and data, sets a new reproducible benchmark for personalized seizure detection. We have also shared a 'plug and play' pipeline to allow other researchers to easily use these algorithms on their own datasets. The success of this competition demonstrates how sharing code and high quality data results in the creation of powerful translational tools with significant potential to impact patient care.