Revisiting crowd behaviour analysis through deep learning: Taxonomy, anomaly detection, crowd emotions, datasets, opportunities and prospects

• Published in: Information fusion Vol. 64; pp. 318 - 335
• Main Authors: Luque Sánchez, Francisco, Hupont, Isabelle, Tabik, Siham, Herrera, Francisco
• Format: Journal Article
• Language: English
• Published: Netherlands Elsevier B.V 01.12.2020; Published by Elsevier B.V
• Subjects: Crowd anomaly detection; Crowd behaviour analysis; Crowd emotions; Deep learning; Full Length; Models fusion; Review; Crowd behaviour analysis; Deep learning; Review; Models fusion; Crowd anomaly detection; Crowd emotions
• ISSN: 1566-2535; 1872-6305
• DOI: 10.1016/j.inffus.2020.07.008

Crowd behaviour analysis is an emerging research area. Due to its novelty, a proper taxonomy to organise its different sub-tasks is still missing. This paper proposes a taxonomic organisation of existing works following a pipeline, where sub-problems in last stages benefit from the results in previous ones. Models that employ Deep Learning to solve crowd anomaly detection, one of the proposed stages, are reviewed in depth, and the few works that address emotional aspects of crowds are outlined. The importance of bringing emotional aspects into the study of crowd behaviour is remarked, together with the necessity of producing real-world, challenging datasets in order to improve the current solutions. Opportunities for fusing these models into already functioning video analytics systems are proposed. •Proposal of hierarchical taxonomy for crowd behaviour analysis subtasks.•Review and numeric comparison of Deep Learning models for crowd anomaly detection.•Discussion of current limitations in datasets and importance of going beyond.•Discussion of the importance of using emotional aspects in crowd behaviour analysis.•Proposals of fusion crowd analysis models into existing video analytics solutions.