A Real-Time Multi-Person Spatio-Temporal Behavior Detection Method in Nighttime Scenes Based on Multi-Model Combination

• Published in: Chinese Control and Decision Conference pp. 2430 - 2434
• Main Authors: Zhou, Jiayi, Wang, Xiaogang, Zhan, Binghan, Wang, Min, Yan, Jiacai
• Format: Conference Proceeding
• Language: English
• Published: IEEE 16.05.2025
• Subjects: Behavioral sciences; Classification algorithms; DeepSort; Location awareness; Multi-person behavior detection; Near-infrared videos; Object detection; Object tracking; Pedestrians; Real-time systems; SlowFast; Surveillance; Thermal analysis; Videos; YOLOv8
• DOI: 10.1109/CCDC65474.2025.11090655

The human spatio-temporal behavior detection in online videos involves detecting human behavior while simultaneously locating the spatio-temporal positions of the individuals. Existing spatio-temporal behavior detection methods are primarily designed to analyze visible light or thermal infrared videos with clear and distinct human features. However, these methods are unsuitable for near-infrared videos that are widely used in monitoring field. Because near-infrared videos have a poor imaging quality and is subject to some interferences, leading to the bad application of tracking and analyzing the online near-infrared video in real-time. Therefore, it is of significant practical value to research methods for the human body's spatio-temporal behavior detection in nearinfrared surveillance videos. Based on a self-build dataset of near-infrared videos for behavior detection, the paper proposes a spatio-temporal behavior detection method for near-infrared surveillance videos using a combination of the YOLOv8n, DeepSort, and SlowFast models. This method aims to achieve multi-person behavior detection in the night scene. The experiments indicate that on the self-build near-infrared surveillance video dataset, the YOLOv8 algorithm model for pedestrian detection at night has a precision of 98.3 %, a recall of 98.2 %, and a mAP@ 0.50 of 99.20 %. The SlowFast algorithm model for near-infrared behavior classification has a mAP@ 0.50 of 80.58%.