Learning Scene-Independent Group Descriptors for Crowd Understanding

• Published in: IEEE transactions on circuits and systems for video technology Vol. 27; no. 6; pp. 1290 - 1303
• Main Authors: Jing Shao, Loy, Chen Change, Xiaogang Wang
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.06.2017; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Circuit stability; Classification; Clips; Conveying; Crowd monitoring; Crowded scene understanding; Feature extraction; Group dynamics; group-property analysis; Hidden Markov models; Learning; Psychology; Public spaces; Retrieval; Robustness; Stability; Stability analysis; video analysis; Video data; Visualization
• ISSN: 1051-8215; 1558-2205
• DOI: 10.1109/TCSVT.2016.2539878

Groups are the primary entities that make up a crowd. Understanding group-level dynamics and properties is thus scientifically important and practically useful in a wide range of applications, especially for crowd understanding. In this paper, we show that fundamental group-level properties, such as intra-group stability and inter-group conflict, can be systematically quantified by visual descriptors. This is made possible through learning a novel collective transition prior, which leads to a robust approach for group segregation in public spaces. From the former, we further devise a rich set of group-property visual descriptors. These descriptors are scene-independent and can be effectively applied to public scenes with a variety of crowd densities and distributions. Extensive experiments on hundreds of public scene video clips demonstrate that such property descriptors are complementary to each other, scene-independent, and they convey critical information on physical states of a crowd. The proposed group-level descriptors show promising results and potentials in multiple applications, including crowd dynamic monitoring, crowd video classification, and crowd video retrieval.