Human-in-the-loop active learning via brain computer interface

• Published in: Annals of mathematics and artificial intelligence Vol. 88; no. 11-12; pp. 1191 - 1205
• Main Authors: Netzer, Eitan, Geva, Amir B.
• Format: Journal Article
• Language: English
• Published: Cham Springer International Publishing 01.12.2020; Springer; Springer Nature B.V
• Subjects: Active learning; Algorithms; Analysis; Annotations; Artificial Intelligence; Artificial neural networks; Cognition; Complex Systems; Computer Science; Computer vision; Deep learning; Electroencephalography; Human-computer interface; Industrial research; Machine learning; Machine vision; Marking; Mathematics; Neural networks; Rapid serial visual presentation; Research projects; S696 Cognition and Neurocomputation; Supervised learning; Deep learning; Transfer Learning; Brain Computer Interface; Human-In-The-Loop; EEG; Clustering; Active Learning
• ISSN: 1012-2443; 1573-7470
• DOI: 10.1007/s10472-020-09689-0

This paper develops and examines an innovative methodology for training an artificial neural network to identify and tag target visual objects in a given database. While the field of Artificial Intelligence in general, and computer vision in particular, has greatly advanced in recent years, fast and efficient methods for tagging (i.e., labeling) visual targets are still lacking. Tagging data is important to train, as it allow to train supervised learning models. However, this is a tiresome task that often creates bottlenecks in academic and industrial research projects. In order to develop an algorithm that improves data tagging processes, this study utilizes the advantages of human cognition and machine learning by combining Brain Computer Interface, Human-In-The-Loop, and Deep Learning. Combining these three fields into one algorithm could enable the rapid annotation of large visual databases that have no prior references and cannot be described as a mathematical optimization function. Human-In-The-Loop is an increasingly researched area that refers to the integration of human feedback in computation processes. At present, computer-based deep learning can only be incorporated in the process of identifying and tagging target objects of interest if a predefined database exists – one that has already been defined by a human user. To reduce the scope of this timely and costly process, our algorithm uses machine learning techniques (i.e., active learning) to minimize the number of target objects a human user needs to identify before the computer can successfully carry out the task independently. In our method, users are connected to electroencephalograms electrodes and shown images using rapid serial visual presentation – a fast method for presenting users with images. Some images are target objects, while others are not. Based on users’ brainwave activity when target objects are shown, the computer learns to identify and tag target objects – already in the learning stage (unlike naïve uniform sampling methods that first require human input, and only then begin the learning stage). As such, our work is proof of concept for the effectiveness of involving humans in the computer’s learning stage, i.e., human-in-the-loop as opposed to the traditional method of humans first tagging the data and the machines then learning and creating a model.