Semantic-Based Surveillance Video Retrieval

• Published in: IEEE transactions on image processing Vol. 16; no. 4; pp. 1168 - 1181
• Main Authors: Weiming Hu, Xie, D., Zhouyu Fu, Wenrong Zeng, Maybank, S.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.04.2007; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Activity models; Algorithms; Applied sciences; Clustering algorithms; Computer Security; Content based retrieval; Database Management Systems; Databases, Factual; Detection, estimation, filtering, equalization, prediction; Exact sciences and technology; Image Interpretation, Computer-Assisted - methods; Image processing; Indexing; Information retrieval; Information Storage and Retrieval - methods; Information theory; Information, signal and communications theory; Layout; Natural Language Processing; Pattern recognition; Queries; Retrieval; semantic-based; Semantics; Signal and communications theory; Signal processing; Signal, noise; Studies; Surveillance; Telecommunications and information theory; Testing; Tracking; Trajectories; Trajectory; User-Computer Interface; Video Recording - methods; video retrieval; Visual; Visual databases; visual surveillance; Performance evaluation; Query; Semantic analysis; Video signal; Remote sensing; video retrieval; visual surveillance; semantic-based; Target detection; Target tracking; Keyword; Video technique; Teletraffic; Information retrieval; Pattern recognition; Object recognition; Algorithm; Hierarchized structure; Calling line identification presentation; First order; Object detection; Activity models; Simultaneity; Remote supervision; Indexing
• ISSN: 1057-7149; 1941-0042
• DOI: 10.1109/TIP.2006.891352

Visual surveillance produces large amounts of video data. Effective indexing and retrieval from surveillance video databases are very important. Although there are many ways to represent the content of video clips in current video retrieval algorithms, there still exists a semantic gap between users and retrieval systems. Visual surveillance systems supply a platform for investigating semantic-based video retrieval. In this paper, a semantic-based video retrieval framework for visual surveillance is proposed. A cluster-based tracking algorithm is developed to acquire motion trajectories. The trajectories are then clustered hierarchically using the spatial and temporal information, to learn activity models. A hierarchical structure of semantic indexing and retrieval of object activities, where each individual activity automatically inherits all the semantic descriptions of the activity model to which it belongs, is proposed for accessing video clips and individual objects at the semantic level. The proposed retrieval framework supports various queries including queries by keywords, multiple object queries, and queries by sketch. For multiple object queries, succession and simultaneity restrictions, together with depth and breadth first orders, are considered. For sketch-based queries, a method for matching trajectories drawn by users to spatial trajectories is proposed. The effectiveness and efficiency of our framework are tested in a crowded traffic scene