Convolutional neural network classifies visual stimuli from cortical response recorded with wide-field imaging in mice

• Published in: Journal of neural engineering Vol. 20; no. 2; pp. 26031 - 26045
• Main Authors: De Luca, Daniela, Moccia, Sara, Lupori, Leonardo, Mazziotti, Raffaele, Pizzorusso, Tommaso, Micera, Silvestro
• Format: Journal Article
• Language: English
• Published: England IOP Publishing 01.04.2023
• Subjects: Algorithms; Animals; Calcium; deep learning; Humans; Mice; Neural Networks, Computer; optic nerve; Visual Cortex - physiology; Visual Fields; visual prostheses; visual stimuli decoding; wide-field imaging; deep learning; wide-field imaging; visual stimuli decoding; optic nerve; visual prostheses
• ISSN: 1741-2560; 1741-2552; 1741-2552
• DOI: 10.1088/1741-2552/acc2e7

Objective. The optic nerve is a good location for a visual neuroprosthesis. It can be targeted when a subject cannot receive a retinal prosthesis and it is less invasive than a cortical implant. The effectiveness of an electrical neuroprosthesis depends on the combination of the stimulation parameters which must be optimized, and an optimization strategy might be performing closed-loop stimulation using the evoked cortical response as feedback. However, it is necessary to identify target cortical activation patterns and to associate the cortical activity with the visual stimuli present in the visual field of the subjects. Visual stimuli decoding should be performed on large areas of the visual cortex, and with a method as translational as possible to shift the study to human subjects in the future. The aim of this work is to develop an algorithm that meets these requirements and can be leveraged to automatically associate a cortical activation pattern with the visual stimulus that generated it. Approach. Three mice were presented with ten different visual stimuli, and their primary visual cortex response was recorded using wide-field calcium imaging. Our decoding algorithm relies on a convolutional neural network (CNN), trained to classify the visual stimuli from the correspondent wide-field images. Several experiments were performed to identify the best training strategy and investigate the possibility of generalization. Main results. The best classification accuracy was 75.38% ± 4.77%, obtained pre-training the CNN on the MNIST digits dataset and fine-tuning it on our dataset. Generalization was possible pre-training the CNN to classify Mouse 1 dataset and fine-tuning it on Mouse 2 and Mouse 3, with accuracies of 64.14% ± 10.81% and 51.53% ± 6.48% respectively. Significance. The combination of wide-field calcium imaging and CNNs can be used to classify the cortical responses to simple visual stimuli and might be a viable alternative to existing decoding methodologies. It also allows us to consider the cortical activation as reliable feedback in future optic nerve stimulation experiments.