Data-Driven Lightweight Interest Point Selection for Large-Scale Visual Search

• Published in: IEEE transactions on multimedia Vol. 20; no. 10; pp. 2774 - 2787
• Main Authors: Gao, Feng, Zhang, Xinfeng, Huang, Yicheng, Luo, Yong, Li, Xiaoming, Duan, Ling-Yu
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.10.2018; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: classification; compact descriptors; Encoding; Feature extraction; Feature recognition; feature selection; interest point selection; Lightweight; Machine learning; MPEG encoders; MPEG-CDVA; MPEG-CDVS; Object recognition; Performance enhancement; Proposals; Redundancy; regression; Searching; Transform coding; Video compression; Visual databases; Visual discrimination; Visual search; Visual task performance; Visualization; Weight reduction
• ISSN: 1520-9210; 1941-0077
• DOI: 10.1109/TMM.2018.2818012

With the explosive increase of images and videos, visual analysis has become an essential technique in dealing with the big visual data, which utilizes the visual feature descriptors to search or recognize the images or frames with target objects or events. Subject to the constraints of resources (e.g., memory, bandwidth, storage, etc.), interest point selection is crucial to generate robust compact descriptors for high-efficiency visual analysis by selecting and aggregating the most discriminative local feature descriptors, which has been demonstrated in the state-of-the-art low bit rate visual search works. In this paper, we propose a data-driven lightweight interest point selection approach to significantly improve the performance of visual search, while ameliorating the efficiency of extracting feature descriptors. Comprehensive experimental results over benchmarks have shown that the proposed interest point selection algorithm has significantly improved image matching and retrieval performance in the completed MPEG Compact Descriptors for Visual Search (CDVS) standard as well as the emerging MPEG Compact Descriptors for Video Analytics (CDVA) standard, say 20% mAP gain by data-driven selection against random selection of interest points. In particular, the presented data-driven interest point selection has been adopted by MPEG-CDVS and MPEG-CDVA as a normative technique to improve the aggregation of handcrafted features, which has contributed to the combination of handcrafted features and deep learning (CNN) features as well.