Temporal-channel convolution with self-attention network for human activity recognition using wearable sensors

• Published in: Knowledge-based systems Vol. 278; p. 110867
• Main Authors: Essa, Ehab, Abdelmaksoud, Islam R.
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 25.10.2023
• Subjects: CNN; HAR; Self-attention; Transformer; HAR; CNN; Transformer; Self-attention
• ISSN: 0950-7051; 1872-7409
• DOI: 10.1016/j.knosys.2023.110867

Human activity recognition (HAR) is an essential task in many applications such as health monitoring, rehabilitation, and sports training. Sensor-based HAR has received increasing attention due to the widespread availability of sensors in daily life. In this paper, we propose two novel architectures, the convolution with self-attention network (CSNet) and the temporal-channel convolution with self-attention network (TCCSNet), for classifying sequences of human activity data from different sensors. CSNet leverages both convolution and self-attention to capture both local and global dependencies in the input data, while TCCSNet exploits both temporal and inter-channel dependencies through two branches of convolutions and self-attentions for extracting time-wise and channel-wise information. The proposed methods are evaluated on seven different sensor-based HAR datasets, namely: MHEALTH, PAMAP2, UTD1, UTD2, WHARF, USC-HAD, and WISDM, using the leave-one-subject-out cross-validation protocol. Our experiments show that the proposed models outperform other modern approaches, such as Transformers and long short-term memory (LSTM) based models. •Novel CSNet and TCCSNet models leverage self-attention for activity recognition.•CSNet captures local patterns & long-range dependencies with convolution & attention.•TCCSNet captures temporal and inter-channel dependencies for enhanced recognition.•Proposed methods outperform state-of-the-art using LOSO protocol on seven datasets.