Smartening E-therapy using Facial Expressions and Deep Learning

• Published in: 2020 2nd International Multidisciplinary Information Technology and Engineering Conference (IMITEC) pp. 1 - 8
• Main Authors: Uzor, Gods Gift G., Vadapalli, Hima B.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 25.11.2020
• Subjects: Computers; Confusion Matrix; Consumer electronics; Convolution Neural Network; Deep learning; E-therapy; F1 Score; Facial expressions evaluation; Medical treatment; MTCNN; Precision; Recall; Transfer learning; Visualization
• DOI: 10.1109/IMITEC50163.2020.9334115

Emotional intelligence finds its application in several fields, and researchers are currently looking to explore the possibility for computers to demonstrate such intelligence. Examining human facial expressions, subject to the activities they carry out at certain times can help improve interactions between humans and computers especially in the era of a digitized society. Communication channels include vocal, body gestures, and facial expressions. Body gestures and facial expressions, as a means of communication, are known to be acquired either involuntarily or voluntarily to lay emphasis on emotions that may not be explicitly expressed via vocal means. Facial expressions are one of the common non-verbal visual cues used by humans in communicating emotions. Facial expressions as a channel to estimate emotions is useful in many applications such as e-learning, online marketing, and e-therapy. E-therapy is regarded as having a healthcare professional to provide mental health services via an electronic medium. There happens to be a range of challenges that could prompt therapy to be administered via electronic channels. This study explores the development of a tool that can facilitate the evaluation of a patient's emotion using their facial expressions during an e-therapy session. Further to evaluating facial expressions, there is a medium provided to estimate the expressions and generate a feedback that can be used by the therapist. Models for facial expression estimation and feedback generation uses deep learning and transfer learning techniques. The initial study was carried out using expression samples obtained from the KDEF and JAFFE databases. The results obtained show a 74.9% and 90.9% accuracy in facial expression classification of images from KDEF and JAFFE databases respectively.