PADS: A Probabilistic Activity Detection Framework for Video Data

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 32; no. 12; pp. 2246 - 2261
• Main Authors: Albanese, M, Chellappa, R, Cuntoor, N, Moscato, V, Picariello, A, Subrahmanian, V S, Udrea, O
• Format: Journal Article
• Language: English
• Published: Los Alamitos, CA IEEE 01.12.2010; IEEE Computer Society; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Application software; applications; Applications and expert knowledge-intensive systems; Applied sciences; Artificial intelligence; Atherosclerosis; Computer science; control theory; systems; Computer vision; Educational institutions; Exact sciences and technology; Human Activities - classification; Humans; Image processing; image processing and computer vision; Image Processing, Computer-Assisted - methods; Image segmentation; Intelligence; Libraries; Models, Statistical; Movement; Packaging; Pattern analysis; Pattern Recognition, Automated - methods; Pattern recognition. Digital image processing. Computational geometry; Probabilistic methods; Probability theory; Programming Languages; Prototypes; Segments; video analysis; Video data; Video sequences; vision and scene understanding; Knowledge engineering; Computer vision; Description language; Probabilistic approach; Image processing; Image interpretation; Expert system; Video signal; Probabilistic language; image processing and computer vision; video analysis; Modeling; Image analysis; Applications and expert knowledge-intensive systems; Image sequence; Scene analysis; vision and scene understanding; Fast algorithm; Pattern analysis; Artificial intelligence; applications
• ISSN: 0162-8828; 1939-3539
• DOI: 10.1109/TPAMI.2010.33

There is now a growing need to identify various kinds of activities that occur in videos. In this paper, we first present a logical language called Probabilistic Activity Description Language (PADL) in which users can specify activities of interest. We then develop a probabilistic framework which assigns to any subvideo of a given video sequence a probability that the subvideo contains the given activity, and we finally develop two fast algorithms to detect activities within this framework. OffPad finds all minimal segments of a video that contain a given activity with a probability exceeding a given threshold. In contrast, the OnPad algorithm examines a video during playout (rather than afterwards as OffPad does) and computes the probability that a given activity is occurring (even if the activity is only partially complete). Our prototype Probabilistic Activity Detection System (PADS) implements the framework and the two algorithms, building on top of existing image processing algorithms. We have conducted detailed experiments and compared our approach to four different approaches presented in the literature. We show that-for complex activity definitions-our approach outperforms all the other approaches.