On-Line Video Event Detection by Constraint Flow

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 36; no. 6; pp. 1174 - 1186
• Main Authors: Kwak, Suha, Han, Bohyung, Han, Joon Hee
• Format: Journal Article
• Language: English
• Published: Los Alamitos, CA IEEE 01.06.2014; IEEE Computer Society; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Activity recognition; Algorithms; Applied sciences; Artificial intelligence; Combinatorial analysis; Computer science; control theory; systems; Constraint flow; Dynamic programming; Event detection; Exact sciences and technology; Heuristic algorithms; Hidden Markov models; Inference algorithms; Logical, boolean and switching functions; On-line systems; Optimization; Parsing algorithms; Pattern analysis; Pattern recognition. Digital image processing. Computational geometry; Preprocessing; Probabilistic logic; Searching; Stochastic processes; Temporal logic; Theoretical computing; Video event detection; temporal logic; dynamic programming; Video event detection; constraint flow; activity recognition; On line; Event detection; Line segment; Video signal; Constraint satisfaction; Activity; Syntactic analysis; Combinatorial optimization; Discrete programming; Dynamic programming; Event structure; Mathematical programming; Script; Computer vision; Event management; Temporal logic; Constrained optimization; Dimension reduction; Reactive system; Image analysis; Experimental result; Scene analysis; Optimal solution; Edge detection
• ISSN: 0162-8828; 1939-3539; 2160-9292
• DOI: 10.1109/TPAMI.2013.245

We present a novel approach in describing and detecting the composite video events based on scenarios, which constrain the configurations of target events by temporal-logical structures of primitive events. We propose a new scenario description method to represent composite events more fluently and efficiently, and discuss an on-line event detection algorithm based on a combinatorial optimization. For this purpose, constraint flow-a dynamic configuration of scenario constraints-is first generated automatically by our scenario parsing algorithm. Then, composite event detection is formulated by a constrained discrete optimization problem, whose objective is to find the best video interpretation with respect to the constraint flow. Although the search space for the optimization problem is prohibitively large, our on-line event detection algorithm based on constraint flow using dynamic programming reduces the search space dramatically, handles preprocessing errors effectively, and guarantees a globally optimal solution. Experimental results on natural videos demonstrate the effectiveness of our algorithm.