Detection of Violent Behavior Using Neural Networks and Pose Estimation

• Published in: IEEE access Vol. 10; pp. 86339 - 86352
• Main Authors: Kwan-Loo, Kevin B., Ortiz-Bayliss, Jose C., Conant-Pablos, Santiago E., Terashima-Marin, Hugo, Rad, P.
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Artificial intelligence; Artificial neural networks; Behavioral sciences; Cameras; Classification; Computer vision; Crime; Feature extraction; frame buffer; Guards; Neural networks; pedestrian detection; Pedestrian safety; Police; Pose estimation; Security; Three dimensional models; tracking; Violent behavior
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2022.3198985

Regarding safety and security, felonies and crimes with physical violence remain a significant problem worldwide. Some solutions for pedestrian safety are guards, police car patrolling, sensors, and security cameras. Nonetheless, these methods react only when the crime takes place. In the worst cases, the damage may be irreversible when it has already occurred. Therefore, numerous methods based on Artificial Intelligence have been proposed to solve this problem. Many approaches to detect violent behavior and action recognition rely on 3D convolutional neural networks (3D-CNNs), spatio-temporal models, long short-term memory networks, pose estimation, among other implementations. However, these approaches work in a limited fashion and have not been adapted to uncontrolled environments. Thus, a significant contribution from this work is the development of an innovative solution model capable of detecting violent behavior. This approach focuses on pedestrian detection, tracking, pose estimation, and neural networks to predict pedestrian behavior in video frames. Our proposal uses a time window frame to extract joint angles, given by the pose estimation algorithm, as features for classifying behavior. Another significant contribution of this work is the creation of a new database, Kranok-NV, with a total of 3,683 normal and violent videos. This database was used to train and test the solution model. For the evaluation, we designed a protocol using 10-fold cross-validation. We obtained an accuracy slightly above 98% on the Kranok-NV database with the implemented solution model. Although the proposed solution model detects violent and normal behavior, it can be easily extended to classify other types of behavior.