Hierarchical Modeling and Adaptive Clustering for Real-Time Summarization of Rush Videos

• Published in: IEEE transactions on multimedia Vol. 11; no. 5; pp. 906 - 917
• Main Authors: Jinchang Ren, Jinchang Ren, Jianmin Jiang, Jianmin Jiang
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.08.2009; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Activity level; adaptive clustering; Applied sciences; Artificial intelligence; Clustering; Clustering algorithms; Clusters; Compressed; Computer science; control theory; systems; Content based retrieval; Data mining; Data processing. List processing. Character string processing; Descriptions; Exact sciences and technology; Formal languages; Gunshot detection systems; hierarchical modelling; Information retrieval; Mathematical models; Memory organisation. Data processing; Multimedia; Object detection; Pattern recognition. Digital image processing. Computational geometry; Semantics; Shot; Software; Speech analysis; Studies; TRECVID; video rushes summarization; Videos; Cluster analysis; Speech analysis; Formal method; Hierarchical classification; Redundancy; Abstract; Competitiveness; Video signal; Unsupervised classification; Data mining; Real time; Modeling; Adaptive method; Hierarchic relation; TRECVID; adaptive clustering; Semantics; Sound analysis; Formal specification; Object detection; hierarchical modelling; Activity level; video rushes summarization; Hierarchical system
• ISSN: 1520-9210; 1941-0077
• DOI: 10.1109/TMM.2009.2021782

In this paper, we provide detailed descriptions of a proposed new algorithm for video summarization, which are also included in our submission to TRECVID'08 on BBC rush summarization. Firstly, rush videos are hierarchically modeled using the formal language technique. Secondly, shot detections are applied to introduce a new concept of V-unit for structuring videos in line with the hierarchical model, and thus junk frames within the model are effectively removed. Thirdly, adaptive clustering is employed to group shots into clusters to determine retakes for redundancy removal. Finally, each most representative shot selected from every cluster is ranked according to its length and sum of activity level for summarization. Competitive results have been achieved to prove the effectiveness and efficiency of our techniques, which are fully implemented in the compressed domain. Our work does not require high-level semantics such as object detection and speech/audio analysis which provides a more flexible and general solution for this topic.