A Constrained Probabilistic Petri Net Framework for Human Activity Detection in Video

• Published in: IEEE transactions on multimedia Vol. 10; no. 6; pp. 982 - 996
• Main Authors: Albanese, M., Chellappa, R., Moscato, V., Picariello, A., Subrahmanian, V.S., Turaga, P., Udrea, O.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.10.2008; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Airports; Algorithm design and analysis; Algorithms; Ambiguity; Applied sciences; Artificial intelligence; Banks; Computer science; control theory; systems; Computer systems and distributed systems. User interface; Data security; Exact sciences and technology; Handles; Human motion; Humans; Machine vision; Monitoring; Pattern recognition. Digital image processing. Computational geometry; Petri nets; Robustness; Security; Software; Studies; Surveillance; Uncertainty; Biometrics; Grain size; Multimedia; Computer vision; Probabilistic approach; Image processing; Petri nets; Video signal; Unfolding; Parking; Modeling; machine vision; Multiagent system; Algorithms; Human activity; Surveillance; Airport; Safety; Fast algorithm; Artificial intelligence; Monitoring; Ambiguity; Petri net
• ISSN: 1520-9210; 1941-0077
• DOI: 10.1109/TMM.2008.2001369

Recognition of human activities in restricted settings such as airports, parking lots and banks is of significant interest in security and automated surveillance systems. In such settings, data is usually in the form of surveillance videos with wide variation in quality and granularity. Interpretation and identification of human activities requires an activity model that a) is rich enough to handle complex multi-agent interactions, b) is robust to uncertainty in low-level processing and c) can handle ambiguities in the unfolding of activities. We present a computational framework for human activity representation based on Petri nets. We propose an extension-Probabilistic Petri Nets (PPN)-and show how this model is well suited to address each of the above requirements in a wide variety of settings. We then focus on answering two types of questions: (i) what are the minimal sub-videos in which a given activity is identified with a probability above a certain threshold and (ii) for a given video, which activity from a given set occurred with the highest probability? We provide the PPN-MPS algorithm for the first problem, as well as two different algorithms (naive PPN-MPA and PPN-MPA) to solve the second. Our experimental results on a dataset consisting of bank surveillance videos and an unconstrained TSA tarmac surveillance dataset show that our algorithms are both fast and provide high quality results.