Towards Classifying Human Circadian Rhythm Using Multiple Modalities

• Published in: International Conference on Affective Computing and Intelligent Interaction and workshops pp. 1 - 8
• Main Authors: Riani, Kais, Sharak, Salem, Das, Kapotaksha, Abouelenien, Mohamed, Burzo, Mihai, Mihalcea, Rada, Elson, John, Maranville, Clay, Prakah-Asante, Kwaku, Manzoor, Waqas
• Format: Conference Proceeding
• Language: English
• Published: IEEE 28.09.2021
• Subjects: action units; circadian rhythm; classification; Data models; Image resolution; Imaging; Linguistics; Machine learning; Multimodal dataset; Sleep; thermal imaging; Visualization
• ISSN: 2156-8111
• DOI: 10.1109/ACII52823.2021.9597391

Autonomous vehicles represent one of the most active technologies currently being developed, with research areas addressing, among others, the modeling of the states and behavioral elements of the occupants. This paper contributes to this line of research by studying the circadian rhythm of individuals using a novel multimodal dataset of 36 subjects consisting of five information channels. These channels include visual, thermal, physiological, linguistic, and background data. Moreover, we propose a framework to explore whether the circadian rhythm can be modeled without continuous monitoring and investigate the hypothesis that multimodal features have a greater propensity for improved performance using data points specific to certain times during the day. Our analysis shows that multimodal fusion can lead to an accuracy of up to 77% on identifying energized and enervated states of the participants. Our findings highlight the validity of our hypothesis and present a novel approach for future research.