Anomaly recognition from surveillance videos using 3D convolution neural network

• Published in: Multimedia tools and applications Vol. 80; no. 12; pp. 18693 - 18716
• Main Authors: Maqsood, Ramna, Bajwa, Usama Ijaz, Saleem, Gulshan, Raza, Rana Hammad, Anwar, Muhammad Waqas
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.05.2021; Springer Nature B.V
• Subjects: Activity recognition; Anomalies; Artificial neural networks; Augmentation; Computer Communication Networks; Computer Science; Convolution; Crime; Data Structures and Information Theory; Datasets; Feature extraction; Machine learning; Multimedia Information Systems; Neural networks; Special Purpose and Application-Based Systems; Surveillance; Traffic accidents; Vandalism; Video; Spatial annotation; 3DConvNets; Anomalous activity recognition; Spatial augmentation
• DOI: 10.1007/s11042-021-10570-3
• ISSN: 1380-7501

Anomalous activity recognition deals with identifying the patterns and events that vary from the normal stream. In a surveillance paradigm, these events range from abuse to fighting and road accidents to snatching, etc. Due to the sparse occurrence of anomalous events, anomalous activity recognition from surveillance videos is a challenging research task. The approaches reported can be generally categorized as handcrafted and deep learning-based. Most of the reported studies address binary classification i.e. anomaly detection from surveillance videos. But these reported approaches did not address other anomalous events e.g. abuse, fight, road accidents, shooting, stealing, vandalism, and robbery, etc. from surveillance videos. Therefore, this paper aims to provide an effective framework for the recognition of different real-world anomalies from videos. This study provides a simple, yet effective approach for learning spatiotemporal features using deep 3-dimensional convolutional networks (3D ConvNets) trained on the University of Central Florida (UCF) Crime video dataset. Firstly, the frame-level labels of the UCF Crime dataset are provided, and then to extract anomalous spatiotemporal features more efficiently a fine-tuned 3D ConvNets is proposed. Findings of the proposed study are twofold 1) There exist specific, detectable, and quantifiable features in UCF Crime video feed that associate with each other 2) Multiclass learning can improve generalizing competencies of the 3D ConvNets by effectively learning frame-level information of dataset and can be leveraged in terms of better results by applying spatial augmentation. The proposed study extracted 3D features by providing frame level information and spatial augmentation to a fine-tuned pre-trained model, namely 3DConvNets. Besides, the learned features are compact enough and the proposed approach outperforms significantly from state of art approaches in terms of accuracy on anomalous activity recognition having 82% AUC.