Video Abnormal Action Recognition Based on Multimodal Heterogeneous Transfer Learning

Human abnormal action recognition is crucial for video understanding and intelligent surveillance. However, the scarcity of labeled data for abnormal human actions often hinders the development of high-performance models. Inspired by the multimodal approach, this paper proposes a novel approach that...

Full description

Saved in:
Bibliographic Details
Published inAdvances in Multimedia Vol. 2024; pp. 1 - 12
Main Authors Huang, Hong-Bo, Zheng, Yao-Lin, Hu, Zhi-Ying
Format Journal Article
LanguageEnglish
Published New York Hindawi 19.01.2024
Hindawi Limited
Wiley
Subjects
Adaptation
Algorithms
Analysis
Artificial intelligence
Datasets
Human activity recognition
Human acts
Human behavior
Knowledge management
Learning
Learning strategies
Methods
Neural networks
Performance enhancement
Semantics
Sensors
User generated content
Video data
Online AccessGet full text

Cover

More Information
Summary:Human abnormal action recognition is crucial for video understanding and intelligent surveillance. However, the scarcity of labeled data for abnormal human actions often hinders the development of high-performance models. Inspired by the multimodal approach, this paper proposes a novel approach that leverages text descriptions associated with abnormal human action videos. Our method exploits the correlation between the text domain and the video domain in the semantic feature space and introduces a multimodal heterogeneous transfer learning framework from the text domain to the video domain. The text of the videos is used for feature encoding and knowledge extraction, and knowledge transfer and sharing are realized in the feature space, which is used to assist in the training of the abnormal action recognition model. The proposed method reduces the reliance on labeled video data, improves the performance of the abnormal human action recognition algorithm, and outperforms the popular video-based models, particularly in scenarios with sparse data. Moreover, our framework contributes to the advancement of automatic video analysis and abnormal action recognition, providing insights for the application of multimodal methods in a broader context.
ISSN:1687-5680
1687-5699
DOI:10.1155/2024/4187991