Classification of multiple emotional states from facial expressions in head-fixed mice using a deep learning-based image analysis

• Published in: PloS one Vol. 18; no. 7; p. e0288930
• Main Authors: Tanaka, Yudai, Nakata, Takuto, Hibino, Hiroshi, Nishiyama, Masaaki, Ino, Daisuke
• Format: Journal Article
• Language: English
• Published: United States Public Library of Science 20.07.2023
• Subjects: Analysis; Animal species; Animals; Artificial intelligence; Biology and Life Sciences; Classification; Datasets; Deep learning; Emotion regulation; Emotional factors; Emotions; Face; Facial expression; Image analysis; Image processing; Machine learning; Machine vision; Medicine and Health Sciences; Social Sciences; Surgery; Vision systems; Japan; United States > US
• ISSN: 1932-6203; 1932-6203
• DOI: 10.1371/journal.pone.0288930

Facial expressions are widely recognized as universal indicators of underlying internal states in most species of animals, thereby presenting as a non-invasive measure for assessing physical and mental conditions. Despite the advancement of artificial intelligence-assisted tools for automated analysis of voluminous facial expression data in human subjects, the corresponding tools for mice still remain limited so far. Considering that mice are the most prevalent model animals for studying human health and diseases, a comprehensive characterization of emotion-dependent patterns of facial expressions in mice could extend our knowledge on the basis of emotions and the related disorders. Here, we present a framework for the development of a deep learning-powered tool for classifying facial expressions in head-fixed mouse. We demonstrate that our machine vision was capable of accurately classifying three different emotional states from lateral facial images in head-fixed mouse. Moreover, we objectively determined how our classifier characterized the differences among the facial images through the use of an interpretation technique called Gradient-weighted Class Activation Mapping. Importantly, our machine vision presumably discerned the data by leveraging multiple facial features. Our approach is likely to facilitate the non-invasive decoding of a variety of emotions from facial images in head-fixed mice.