Cross-Modal Attention for Multimodal Information Fusion: A Novel Approach to Attention Deficit Hyperactivity Disorder Detection

This paper presents a novel method for differentiating Attention Deficit Hyperactivity Disorder subjects from control participants by multimodal data fusion, including video observations and questionnaire responses. By exploiting the well known Video Vision Transformer model, we analyse the video mo...

Full description

Saved in:
Bibliographic Details
Published in2024 27th International Conference on Information Fusion (FUSION) pp. 1 - 6
Main Authors Nash, Christian, Nair, Rajesh, Naqvi, Syed Mohsen
Format Conference Proceeding
LanguageEnglish
Published ISIF 08.07.2024
Subjects
Accuracy
Adaptation models
Analytical models
Attention Deficit Hyperactivity Disorder
Attention mechanisms
Data mining
Data models
Deep Learning
Feature extraction
Machine Learning
Mental Health
Multimodal
Training
Transformers
Visualization
Online AccessGet full text

Cover

More Information
Summary:This paper presents a novel method for differentiating Attention Deficit Hyperactivity Disorder subjects from control participants by multimodal data fusion, including video observations and questionnaire responses. By exploiting the well known Video Vision Transformer model, we analyse the video modality to identify the complex spatial-temporal information of ADHD symptoms. Simultaneously, a Multi-Layer Perceptron model is applied to evaluate structured questionnaire data by capturing key cognitive and emotional indicators of the ADHD symptoms. To fuse the two modalities, a cross-modal attention mechanism assigns adaptive weights to each feature based on its classification relevance. The targeted weighting significantly refines the proposed model's decision-making capability by concentrating on the most critical elements of the aggregated information. For training and testing, our novel Multimodal ADHD dataset recorded under the Intelligent Sensing ADHD Trial in collaboration with Cumbria, Northumberland, Tyne and Wear NHS Foundation Trust UK is evaluated. The proposed model, ADViQ-AL achieves a 98.18% classification accuracy, 97.83% sensitivity, and 98.53% specificity in classifying ADHD and control groups.
DOI:10.23919/FUSION59988.2024.10706381