Intelligent video anomaly detection and classification using faster RCNN with deep reinforcement learning model

• Published in: Image and vision computing Vol. 112; p. 104229
• Main Authors: Mansour, Romany F., Escorcia-Gutierrez, José, Gamarra, Margarita, Villanueva, Jair A., Leal, Nallig
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.08.2021
• Subjects: Anomaly detection; Deep reinforcement learning; Intelligent systems; UCSD dataset; Video surveillance; UCSD dataset; Video surveillance; Deep reinforcement learning; Intelligent systems; Anomaly detection
• ISSN: 0262-8856; 1872-8138
• DOI: 10.1016/j.imavis.2021.104229

Recently, intelligent video surveillance applications have become essential in public security by the use of computer vision technologies to investigate and understand long video streams. Anomaly detection and classification are considered a major element of intelligent video surveillance. The aim of anomaly detection is to automatically determine the existence of abnormalities in a short time period. Deep reinforcement learning (DRL) techniques can be employed for anomaly detection, which integrates the concepts of reinforcement learning and deep learning enabling the artificial agents in learning the knowledge and experience from actual data directly. With this motivation, this paper presents an Intelligent Video Anomaly Detection and Classification using Faster RCNN with Deep Reinforcement Learning Model, called IVADC-FDRL model. The presented IVADC-FDRL model operates on two major stages namely anomaly detection and classification. Firstly, Faster RCNN model is applied as an object detector with Residual Network as a baseline model, which detects the anomalies as objects. Besides, deep Q-learning (DQL) based DRL model is employed for the classification of detected anomalies. In order to validate the effective anomaly detection and classification performance of the IVADC-FDRL model, an extensive set of experimentations were carried out on the benchmark UCSD anomaly dataset. The experimental results showcased the better performance of the IVADC-FDRL model over the other compared methods with the maximum accuracy of 98.50% and 94.80% on the applied Test004 and Test007 dataset respectively. •Design an intelligent IVADC-FDRL model for video anomaly detection and classification.•Propose a Faster RCNN with ResNet as a baseline model for Anomaly Detection.•Employ a deep Q-learning (DQL) model to classify detected anomalies in video frames.•Validate the anomaly detection and classification performance on UCSD Anomaly dataset.•Proposed model achieves a maximum accuracy of 98.50% on the applied Test004 dataset.