SFF-DA: Spatiotemporal Feature Fusion for Nonintrusively Detecting Anxiety

• Published in: IEEE transactions on instrumentation and measurement Vol. 73; p. 1
• Main Authors: Mo, Haimiao, Li, Yuchen, Han, Peng, Liao, Xiao, Zhang, Wei, Ding, Shuai
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.01.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Anxiety; Anxiety disorders; Data mining; Deep learning; Facial Video Understanding; Feature extraction; Feature Fusion; Few-Shot Learning; Machine learning; Mental disorders; Mouth; Nonintrusive Anxiety Detection; Physiology; Signs and symptoms; Spatiotemporal Feature Extraction; Spatiotemporal phenomena; Training
• DOI: 10.1109/TIM.2023.3341132
• ISSN: 0018-9456

The early detection of anxiety disorders is crucial in mitigating distress and enhancing outcomes for individuals with mental disorders. Deep learning methods and traditional machine learning approaches are both used for the early screening of mental disorders, particularly those with anxiety symptoms. These methods excel at extracting spatiotemporal features associated with mental disorders; however, they often overlook potential interrelationships among these features. Furthermore, the effectiveness of existing methods is hindered by disparities in the quality of subject data collected in nonlaboratory settings, limited data sample sizes, and other factors. Therefore, we propose a non-intrusive anxiety detection framework based on spatiotemporal feature fusion. Within this framework, spatiotemporal features are extracted from physiological and behavioural data through a shared feature extraction network. Additionally, we design a few-shot learning architecture to compute the coupling of fused spatiotemporal features, assessing the similarity of various feature types within sample pairs. Furthermore, joint training strategies applied within the framework significantly enhance the performance of classification performance. We validate the performance of our framework through experiments with a real-world seafarer dataset. The experimental results unequivocally demonstrate that our framework outperforms comparative approaches.