Purposive Hidden-Object-Game: Embedding Human Computation in Popular Game

• Published in: IEEE transactions on multimedia Vol. 14; no. 5; pp. 1496 - 1507
• Main Authors: Jiashi Feng, Yuzhao Ni, Jian Dong, Zilei Wang, Shuicheng Yan
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.10.2012; Institute of Electrical and Electronics Engineers
• Subjects: Accuracy; Applied sciences; Artificial intelligence; Computer science; control theory; systems; Computer systems and distributed systems. User interface; Exact sciences and technology; Games; Games with a purpose (GWAP); human computing; Humans; image processing; Information systems. Data bases; Labeling; Memory organisation. Data processing; Multimedia communication; multimedia computing; Noise measurement; Pattern recognition. Digital image processing. Computational geometry; Semantics; Software; Games with a purpose (GWAP); Entertainment; Image processing; Algorithmics; Very large databases; Labelling; multimedia computing; Interest; Cheating; Semantics; Database; Learning algorithm; Object location; Multimedia; Computer vision; Refinement method; Image retrieval; Social network; Object recognition; Game theory; Computer games; Annotation; human computing; Artificial intelligence; Indexing
• ISSN: 1520-9210; 1941-0077
• DOI: 10.1109/TMM.2012.2198801

Having sufficient training images with fully annotated object locations is undoubtedly critical for modern learning-based image annotation, retrieval, and object detection methods. Typically, collecting such annotations for large-scale datasets is notoriously tedious because the process involves amount of manual cropping and hand labeling operations. In this work, following the principle of games with a purpose (GWAP), we design a so-called purposive hidden-object-game (P-HOG), which imperceptibly embeds localizing objects into enjoyable playing game process and thus attracts many people to make voluntary contribution to annotating images. In particular, besides preserving the interestingness as popular HOG games, P-HOG is able to automatically generate satisfactory game images (i.e., "hide" certain items into target images) by integrating several semantic and visual processing techniques. P-HOG is also built in an effective mechanism to prevent the players from cheating. The mechanism inherits the merit of Recaptcha and identifies potential cheating behavior based on the annotation accuracy of some known items. Moreover, P-HOG will filter noisy annotations effectively based on a weighted majority method and improve the accuracy of the raw annotations from the players. Most importantly, players only play P-HOG for entertainment purpose and they are unaware of the background data collection procedure. The collected data are used towards constructing a large database, which may benefit general learning-based algorithms for multimedia tasks. To the best of our knowledge, this is the first work dedicated to such a specific and important task under the GWAP framework. We conduct a pilot study of the game prototype and the comprehensive experiments show that the P-HOG appeals to general players, and is effective for collecting massive object locations with satisfactory accuracy, which further boosts the algorithmic performances for both tag refinement and image annotation tasks.