Robust Web Image Annotation via Exploring Multi-Facet and Structural Knowledge

• Published in: IEEE transactions on image processing Vol. 26; no. 10; pp. 4871 - 4884
• Main Authors: Hu, Mengqiu, Yang, Yang, Shen, Fumin, Zhang, Luming, Shen, Heng Tao, Li, Xuelong
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.10.2017; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: <italic xmlns:ali="http://www.niso.org/schemas/ali/1.0/" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">p -norm; Annotations; Datum (elevation); Digital imaging; Image annotation; Image management; Indexing; Iterative algorithms; Iterative methods; Labels; Machine learning; Manifolds; multi-view learning; Multimedia; Multimedia communication; Robustness; Semantics; semi-supervised learning; Semisupervised learning; Supervised learning
• ISSN: 1057-7149; 1941-0042
• DOI: 10.1109/TIP.2017.2717185

Driven by the rapid development of Internet and digital technologies, we have witnessed the explosive growth of Web images in recent years. Seeing that labels can reflect the semantic contents of the images, automatic image annotation, which can further facilitate the procedure of image semantic indexing, retrieval, and other image management tasks, has become one of the most crucial research directions in multimedia. Most of the existing annotation methods, heavily rely on well-labeled training data (expensive to collect) and/or single view of visual features (insufficient representative power). In this paper, inspired by the promising advance of feature engineering (e.g., CNN feature and scale-invariant feature transform feature) and inexhaustible image data (associated with noisy and incomplete labels) on the Web, we propose an effective and robust scheme, termed robust multi-view semi-supervised learning (RMSL), for facilitating image annotation task. Specifically, we exploit both labeled images and unlabeled images to uncover the intrinsic data structural information. Meanwhile, to comprehensively describe an individual datum, we take advantage of the correlated and complemental information derived from multiple facets of image data (i.e., multiple views or features). We devise a robust pairwise constraint on outcomes of different views to achieve annotation consistency. Furthermore, we integrate a robust classifier learning component via ℓ 2,p loss, which can provide effective noise identification power during the learning process. Finally, we devise an efficient iterative algorithm to solve the optimization problem in RMSL. We conduct comprehensive experiments on three different data sets, and the results illustrate that our proposed approach is promising for automatic image annotation.