Unsupervised video summarization via clustering validity index

• Published in: Multimedia tools and applications Vol. 79; no. 45-46; pp. 33417 - 33430
• Main Authors: Zhao, Ye, Guo, Yanrong, Sun, Rui, Liu, Zhengqiong, Guo, Dan
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.12.2020; Springer Nature B.V
• Subjects: Clustering; Computer Communication Networks; Computer Science; Data Structures and Information Theory; Multimedia Information Systems; Segmentation; Special Purpose and Application-Based Systems; Validity; Video data; Clustering validity index; Representative selection; Video summarization; Frame selection
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-019-7582-8

Although lots of the prior works have been proposed to solve the representative selection problem of video summarization, the main difficulty is still left for determining the optimal representatives’ number of the raw videos that are not annotated. In this paper, we propose an unsupervised video summarization method by motion-based frame selection and a novel clustering validity indexes to determine the optimal representatives of the original video. The proposed framework segments shots and selects candidate frames by evaluating their forward and backward motion and can automatically select representatives to highlight all the significant visual properties. Shots are segmented uniformly and the frame with the largest motion is extracted in each segmentation to form the video candidate frame subset. Then Affinity Propagation combined with the validity index is used to automatically select the optimal representatives from the candidate frame subset. Our experimental result on several benchmark datasets demonstrates the robustness and effectiveness of our proposed method.