Fusion of CCTV Video and Spatial Information for Automated Crowd Congestion Monitoring in Public Urban Spaces

• Published in: Algorithms Vol. 16; no. 3; p. 154
• Main Authors: Wong, Vivian, Law, Kincho
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01.03.2023
• Subjects: Algorithms; Automation; Case studies; Closed circuit television; Computer vision; Congestion; crowd congestion; Crowd monitoring; Crowd stampedes; Datasets; Deep learning; Floorplans; Forecasts and trends; Geometric transformation; graph representation learning; Kalman filters; Machine vision; Mechanization; Multisensor fusion; Public safety; Public spaces; Railway stations; Sensors; Spatial data; Surveillance; Urban areas; Video data; Visual aids; New York; United States > US
• ISSN: 1999-4893; 1999-4893
• DOI: 10.3390/a16030154

Crowd congestion is one of the main causes of modern public safety issues such as stampedes. Conventional crowd congestion monitoring using closed-circuit television (CCTV) video surveillance relies on manual observation, which is tedious and often error-prone in public urban spaces where crowds are dense, and occlusions are prominent. With the aim of managing crowded spaces safely, this study proposes a framework that combines spatial and temporal information to automatically map the trajectories of individual occupants, as well as to assist in real-time congestion monitoring and prediction. Through exploiting both features from CCTV footage and spatial information of the public space, the framework fuses raw CCTV video and floor plan information to create visual aids for crowd monitoring, as well as a sequence of crowd mobility graphs (CMGraphs) to store spatiotemporal features. This framework uses deep learning-based computer vision models, geometric transformations, and Kalman filter-based tracking algorithms to automate the retrieval of crowd congestion data, specifically the spatiotemporal distribution of individuals and the overall crowd flow. The resulting collective crowd movement data is then stored in the CMGraphs, which are designed to facilitate congestion forecasting at key exit/entry regions. We demonstrate our framework on two video data, one public from a train station dataset and the other recorded at a stadium following a crowded football game. Using both qualitative and quantitative insights from the experiments, we demonstrate that the suggested framework can be useful to help assist urban planners and infrastructure operators with the management of congestion hazards.