Quality-guided key frames selection from video stream based on object detection

• Published in: Journal of visual communication and image representation Vol. 65; p. 102678
• Main Authors: Chen, Mingju, Han, Xiaofeng, Zhang, Hua, Lin, Guojun, Kamruzzaman, M.M.
• Format: Journal Article
• Language: English
• Published: Elsevier Inc 01.12.2019
• Subjects: Convolutional neural network; Key frame; Object detection; Quality assessment model; SIFT characteristics; Quality assessment model; SIFT characteristics; Convolutional neural network; Key frame; Object detection
• ISSN: 1047-3203; 1095-9076
• DOI: 10.1016/j.jvcir.2019.102678

Object detection technique is widely applied in modern intelligent systems, such as pedestrian tracking, video surveillance. Key frames selection aims to select more informative frames and reduce amount of redundant information frames. Traditional methods leveraged SIFT feature, which have high key frame selection error rate. In this paper, we propose a novel key frames selection method based on object detection and image quality. Specifically, we first leverage object detector to detect object, such as pedestrian, vehicles. Then, each training frame will be assigned with a quality score, where frames contain objects have high quality score. Afterwards, we leverage CNN based AlexNet architecture for deep feature representation extraction. Our algorithm combines mutual information entropy and SURF image local features to extract key frames. Comprehensive experiments verify the feasibility of practicing the key frame extractor based on convolutional neural network by training the model, and conduct a quality assessment model study.