Deep convolutional framework for abnormal behavior detection in a smart surveillance system

• Published in: Engineering applications of artificial intelligence Vol. 67; pp. 226 - 234
• Main Authors: Ko, Kwang-Eun, Sim, Kwee-Bo
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.01.2018
• Subjects: Behavior recognition; Convolutional neural network; Long short-term memory; Smart surveillance system; 99-00; Long short-term memory; Smart surveillance system; 00-01; Behavior recognition; Convolutional neural network
• ISSN: 0952-1976; 1873-6769
• DOI: 10.1016/j.engappai.2017.10.001

The ability to instantly detect risky behavior in video surveillance systems is a critical issue in a smart surveillance system. In this paper, a unified framework based on a deep convolutional framework is proposed to detect abnormal human behavior from a standard RGB image. The objective of the unified structure is to improve detection speed while maintaining recognition accuracy. The deep convolutional framework consists of (1) a human subject detection and discrimination module that is proposed to solve the problem of separating object entities, in contrast to previous object detection algorithms, (2) a posture classification module to extract spatial features of abnormal behavior, and (3) an abnormal behavior detection module based on long short-term memory (LSTM). Experiments on a benchmark dataset evaluate the potential of the proposed method in the context of smart surveillance. The results indicate that the proposed method provides satisfactory performance in detecting abnormal behavior in a real-world scenario. •We propose a method for discriminating objects within a same category by using deep learning based object detection algorithm and Kalman filter.•We propose a deep convolutional framework for extracting dynamic feature of human behavior from standard RGB image.•We model the behavior goal estimation process using a long-short term memory.•We evaluate the method for simple and complex behaviors with goal-directed objects.