Fall event detection with global and temporal local information in real-world videos

• Published in: Multimedia tools and applications Vol. 81; no. 5; pp. 6943 - 6956
• Main Authors: Pang, Wenfeng, He, Qianhua, Chen, Yuanfeng, Li, Yanxiong
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.02.2022; Springer Nature B.V
• Subjects: Cameras; Computer Communication Networks; Computer Science; Data Structures and Information Theory; Datasets; Injury prevention; Mathematical analysis; Modules; Multimedia; Multimedia Information Systems; Neural networks; Robots; Special Purpose and Application-Based Systems; Surveillance; Temporal local information; Real-world fall dataset; Fall event detection; Multiple instance module
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-022-12018-8

Fall event detection is a task to determine whether a video includes falls, which can offer timely alarms to help reduce fall injury outcomes. This study applied typical 3D-convoultional network (C3D) as the classifier, but the basic C3D tends to extract the global information ignoring frame-wise information, whereas fall events are usually instantaneous. In the case that frame-level labels are difficult to obtain, we treat the problem of extracting temporal local information as a semi-supervised task, and introduce multiple instance (MI) module to construct an additional branch in C3D, which utilizes the multiple instance learning (MIL) to focus on smaller time-scale information. The MI module chooses a segment most likely to contain falls in a video for calculating the MIL loss, which is combined with the original cross-entropy loss as the total loss. In view of the fact fall events in the existing datasets are insufficient and not representative of real-world environments, we built a new dataset called Real-World Fall (RWF) providing a large number of factual fall cases. Experimental results on the RWF dataset indicated that our proposed model achieves superior performance by simultaneously extracting global and temporal local information. Further studies were conducted to assessed factors effect in MI module, including loss functions and pooling methods.