Drowsiness Level Modeling Based on Facial Skin Temperature Distribution Using a Convolutional Neural Network

• Published in: IEEJ transactions on electrical and electronic engineering Vol. 14; no. 6; pp. 870 - 876
• Main Authors: Adachi, Hiroko, Oiwa, Kosuke, Nozawa, Akio
• Format: Journal Article
• Language: English
• Published: Hoboken, USA John Wiley & Sons, Inc 01.06.2019; Wiley Subscription Services, Inc
• Subjects: Artificial neural networks; convolutional neural network; deep learning; drowsiness detection; facial skin temperature; Feature extraction; Feature maps; Industrial applications; Neural networks; Skin temperature; Temperature distribution; thermal image; Traffic accidents
• ISSN: 1931-4973; 1931-4981
• DOI: 10.1002/tee.22876

Several drowsiness detection technologies have been developed to combat traffic accidents. Drowsiness evaluation has been attempted using the time‐series changes in nasal skin temperature. However, constructing a detection model based on this has been difficult because recent studies have reported an individual difference in skin temperature behaviors. In this study, the conventional method of extracting features was revised. The model of the level of drowsiness was constructed based on facial skin temperature distribution using a convolutional neural network (CNN). The drowsiness level was developed by the New Energy and Industrial Technology Development Organization as an objective drowsiness evaluation index based on facial expressions. With CNNs, features related to learning can be observed as feature quantity distributions. As a result, a general model created has a lack of generality and it is thought that not only the response to drowsiness, but also face shape, exhibit individual differences. Consequently, different features were found in each subject. Through feature maps in individual models, it is believed that skin temperature changes have both reproducible and individual response characteristics to drowsiness, due to the asymmetric left–right change in feature quantity distribution depending on the observed drowsiness level. This method was compared with the conventional method of extracting other features related to drowsiness. It suggested that the skin temperature of not only the nasal region but also the entire face changes as drowsiness increases. Consequently, each discrimination rate calculated by the CNN was at least 20% higher than that obtained via conventional methods. © 2019 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.