A zero-shot learning approach to the development of brain-computer interfaces for image retrieval

Brain decoding-the process of inferring a person's momentary cognitive state from their brain activity-has enormous potential in the field of human-computer interaction. In this study we propose a zero-shot EEG-to-image brain decoding approach which makes use of state-of-the-art EEG preprocessi...

Full description

Saved in:
Bibliographic Details
Published inPloS one Vol. 14; no. 9; p. e0214342
Main Authors McCartney, Ben, Martinez-Del-Rincon, Jesus, Devereux, Barry, Murphy, Brian
Format Journal Article
LanguageEnglish
Published United States Public Library of Science 16.09.2019
Public Library of Science (PLoS)
Subjects
Accuracy
Analysis
Biology and Life Sciences
Brain
Brain - physiology
Brain research
Brain-Computer Interfaces
Classification
Cognitive ability
Computer and Information Sciences
Computer vision
Decoding
EEG
Electroencephalography
Electroencephalography - methods
Engineering and Technology
Human-computer interaction
Human-computer interface
Humans
Image classification
Image management
Image retrieval
Information processing
Interfaces
Learning
Machine Learning
Medicine and Health Sciences
Methods
Neural circuitry
Neural networks
Neuroimaging
Neurosciences
Prostheses
Rapid serial visual presentation
Research and Analysis Methods
Researchers
Retrieval
Semantics
Social Sciences
Visual Perception
Zero-shot learning
United Kingdom
United Kingdom > UK
Online AccessGet full text

Cover

More Information
Summary:Brain decoding-the process of inferring a person's momentary cognitive state from their brain activity-has enormous potential in the field of human-computer interaction. In this study we propose a zero-shot EEG-to-image brain decoding approach which makes use of state-of-the-art EEG preprocessing and feature selection methods, and which maps EEG activity to biologically inspired computer vision and linguistic models. We apply this approach to solve the problem of identifying viewed images from recorded brain activity in a reliable and scalable way. We demonstrate competitive decoding accuracies across two EEG datasets, using a zero-shot learning framework more applicable to real-world image retrieval than traditional classification techniques.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
Competing Interests: During the course of this study one of the authors (Brian Murphy) has been under the employ of BrainWaveBank Ltd. as CSO. Although Brian is serving as the point of contact for access to one of the EEG datasets we analyse in this study, this dataset was created prior to the founding of BrainWaveBank and as such the funder has no association with the dataset. At this time Brian was under the employ of the University of Trento. This does not alter our adherence to PLOS ONE policies on sharing data and materials.
ISSN:1932-6203
1932-6203
DOI:10.1371/journal.pone.0214342