Three-Stage Deep Learning Framework for Video Surveillance

• Published in: Applied sciences Vol. 14; no. 1; p. 408
• Main Authors: Lee, Ji-Woon, Kang, Hyun-Soo
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01.01.2024
• Subjects: Accuracy; Analysis; anomaly detection; Automation; Computational linguistics; Criminal investigations; Deep learning; Language processing; Natural language interfaces; Neural networks; RNN; Safety and security measures; Surveillance; Transformers; video surveillance; Violence; Germany
• ISSN: 2076-3417; 2076-3417
• DOI: 10.3390/app14010408

The escalating use of security cameras has resulted in a surge in images requiring analysis, a task hindered by the inefficiency and error-prone nature of manual monitoring. In response, this study delves into the domain of anomaly detection in CCTV security footage, addressing challenges previously encountered in analyzing videos with complex or dynamic backgrounds and long sequences. We introduce a three-stage deep learning architecture designed to detect abnormalities in security camera videos. The first stage employs a pre-trained convolutional neural network to extract features from individual video frames. Subsequently, these features are transformed into time series data in the second stage, utilizing a blend of bidirectional long short-term memory and multi-head attention to analyze short-term frame relationships. The final stage leverages relative positional embeddings and a custom Transformer encoder to interpret long-range frame relationships and identify anomalies. Tested on various open datasets, particularly those with complex backgrounds and extended sequences, our method demonstrates enhanced accuracy and efficiency in video analysis. This approach not only improves current security camera analysis but also shows potential for diverse application settings, signifying a significant advancement in the evolution of security camera monitoring and analysis technologies.