Fear Detection in Multimodal Affective Computing: Physiological Signals versus Catecholamine Concentration

• Published in: Sensors (Basel, Switzerland) Vol. 22; no. 11; p. 4023
• Main Authors: Gutiérrez-Martín, Laura, Romero-Perales, Elena, de Baranda Andújar, Clara Sainz, F Canabal-Benito, Manuel, Rodríguez-Ramos, Gema Esther, Toro-Flores, Rafael, López-Ongil, Susana, López-Ongil, Celia
• Format: Journal Article
• Language: English
• Published: Switzerland MDPI AG 26.05.2022; MDPI
• Subjects: Adrenal glands; Affective computing; Artificial intelligence; Audio data; Audio signals; Blood; Blood volume; Brain research; Catecholamine; Catecholamines; Classification; Dopamine; Electromyography; emotion classification; Emotions; Fear; Fear & phobias; Hormones; Hypotheses; multimodal affective computing; Noradrenaline; Physiology; Plasma; Respiration; Signal monitoring; Skin; Skin temperature; Software; Tumors; Variables; Virtual reality; wearable devices; Wearable technology; wearable devices; catecholamines; emotion classification; multimodal affective computing
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s22114023

Affective computing through physiological signals monitoring is currently a hot topic in the scientific literature, but also in the industry. Many wearable devices are being developed for health or wellness tracking during daily life or sports activity. Likewise, other applications are being proposed for the early detection of risk situations involving sexual or violent aggressions, with the identification of panic or fear emotions. The use of other sources of information, such as video or audio signals will make multimodal affective computing a more powerful tool for emotion classification, improving the detection capability. There are other biological elements that have not been explored yet and that could provide additional information to better disentangle negative emotions, such as fear or panic. Catecholamines are hormones produced by the adrenal glands, two small glands located above the kidneys. These hormones are released in the body in response to physical or emotional stress. The main catecholamines, namely adrenaline, noradrenaline and dopamine have been analysed, as well as four physiological variables: skin temperature, electrodermal activity, blood volume pulse (to calculate heart rate activity. i.e., beats per minute) and respiration rate. This work presents a comparison of the results provided by the analysis of physiological signals in reference to catecholamine, from an experimental task with 21 female volunteers receiving audiovisual stimuli through an immersive environment in virtual reality. Artificial intelligence algorithms for fear classification with physiological variables and plasma catecholamine concentration levels have been proposed and tested. The best results have been obtained with the features extracted from the physiological variables. Adding catecholamine's maximum variation during the five minutes after the video clip visualization, as well as adding the five measurements (1-min interval) of these levels, are not providing better performance in the classifiers.