CNN features with bi-directional LSTM for real-time anomaly detection in surveillance networks

• Published in: Multimedia tools and applications Vol. 80; no. 11; pp. 16979 - 16995
• Main Authors: Ullah, Waseem, Ullah, Amin, Haq, Ijaz Ul, Muhammad, Khan, Sajjad, Muhammad, Baik, Sung Wook
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.05.2021; Springer Nature B.V
• Subjects: Anomalies; Artificial neural networks; Complexity; Computer Communication Networks; Computer Science; Computer vision; Crime; Data Structures and Information Theory; Datasets; Decision making; Feature extraction; Frames (data processing); Multilayers; Multimedia Information Systems; Public safety; Special Purpose and Application-Based Systems; Surveillance; Surveillance systems; Video data; Deep learning; Intelligent surveillance networks; LSTM; Smart surveillance; Crime detection; Anomaly detection
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-020-09406-3

In current technological era, surveillance systems generate an enormous volume of video data on a daily basis, making its analysis a difficult task for computer vision experts. Manually searching for unusual events in these massive video streams is a challenging task, since they occur inconsistently and with low probability in real-world surveillance. In contrast, deep learning-based anomaly detection reduces human labour and its decision making ability is comparatively reliable, thereby ensuring public safety. In this paper, we present an efficient deep features-based intelligent anomaly detection framework that can operate in surveillance networks with reduced time complexity. In the proposed framework, we first extract spatiotemporal features from a series of frames by passing each one to a pre-trained Convolutional Neural Network (CNN) model. The features extracted from the sequence of frames are valuable in capturing anomalous events. We then pass the extracted deep features to multi-layer Bi-directional Long Short-term Memory (BD-LSTM) model, which can accurately classify ongoing anomalous/normal events in complex surveillance scenes of smart cities. We performed extensive experiments on various anomaly detection benchmark datasets to validate the functionality of the proposed framework within complex surveillance scenarios. We reported a 3.41% and 8.09% increase in accuracy on UCF-Crime and UCFCrime2Local datasets compared to state-of-the-art methods.