Purposive Hidden-Object-Game: Embedding Human Computation in Popular Game

Having sufficient training images with fully annotated object locations is undoubtedly critical for modern learning-based image annotation, retrieval, and object detection methods. Typically, collecting such annotations for large-scale datasets is notoriously tedious because the process involves amo...

Full description

Saved in:
Bibliographic Details
Published inIEEE transactions on multimedia Vol. 14; no. 5; pp. 1496 - 1507
Main Authors Jiashi Feng, Yuzhao Ni, Jian Dong, Zilei Wang, Shuicheng Yan
Format Journal Article
LanguageEnglish
Published New York, NY IEEE 01.10.2012
Institute of Electrical and Electronics Engineers
Subjects
Online AccessGet full text

Cover

Loading…
Abstract Having sufficient training images with fully annotated object locations is undoubtedly critical for modern learning-based image annotation, retrieval, and object detection methods. Typically, collecting such annotations for large-scale datasets is notoriously tedious because the process involves amount of manual cropping and hand labeling operations. In this work, following the principle of games with a purpose (GWAP), we design a so-called purposive hidden-object-game (P-HOG), which imperceptibly embeds localizing objects into enjoyable playing game process and thus attracts many people to make voluntary contribution to annotating images. In particular, besides preserving the interestingness as popular HOG games, P-HOG is able to automatically generate satisfactory game images (i.e., "hide" certain items into target images) by integrating several semantic and visual processing techniques. P-HOG is also built in an effective mechanism to prevent the players from cheating. The mechanism inherits the merit of Recaptcha and identifies potential cheating behavior based on the annotation accuracy of some known items. Moreover, P-HOG will filter noisy annotations effectively based on a weighted majority method and improve the accuracy of the raw annotations from the players. Most importantly, players only play P-HOG for entertainment purpose and they are unaware of the background data collection procedure. The collected data are used towards constructing a large database, which may benefit general learning-based algorithms for multimedia tasks. To the best of our knowledge, this is the first work dedicated to such a specific and important task under the GWAP framework. We conduct a pilot study of the game prototype and the comprehensive experiments show that the P-HOG appeals to general players, and is effective for collecting massive object locations with satisfactory accuracy, which further boosts the algorithmic performances for both tag refinement and image annotation tasks.
AbstractList Having sufficient training images with fully annotated object locations is undoubtedly critical for modern learning-based image annotation, retrieval, and object detection methods. Typically, collecting such annotations for large-scale datasets is notoriously tedious because the process involves amount of manual cropping and hand labeling operations. In this work, following the principle of games with a purpose (GWAP), we design a so-called purposive hidden-object-game (P-HOG), which imperceptibly embeds localizing objects into enjoyable playing game process and thus attracts many people to make voluntary contribution to annotating images. In particular, besides preserving the interestingness as popular HOG games, P-HOG is able to automatically generate satisfactory game images (i.e., "hide" certain items into target images) by integrating several semantic and visual processing techniques. P-HOG is also built in an effective mechanism to prevent the players from cheating. The mechanism inherits the merit of Recaptcha and identifies potential cheating behavior based on the annotation accuracy of some known items. Moreover, P-HOG will filter noisy annotations effectively based on a weighted majority method and improve the accuracy of the raw annotations from the players. Most importantly, players only play P-HOG for entertainment purpose and they are unaware of the background data collection procedure. The collected data are used towards constructing a large database, which may benefit general learning-based algorithms for multimedia tasks. To the best of our knowledge, this is the first work dedicated to such a specific and important task under the GWAP framework. We conduct a pilot study of the game prototype and the comprehensive experiments show that the P-HOG appeals to general players, and is effective for collecting massive object locations with satisfactory accuracy, which further boosts the algorithmic performances for both tag refinement and image annotation tasks.
Author Jian Dong
Zilei Wang
Shuicheng Yan
Yuzhao Ni
Jiashi Feng
Author_xml – sequence: 1
  surname: Jiashi Feng
  fullname: Jiashi Feng
  email: a0066331@nus.edu.sg
  organization: Dept. of Electr. & Comput. Eng., Nat. Univ. of Singapore, Singapore, Singapore
– sequence: 2
  surname: Yuzhao Ni
  fullname: Yuzhao Ni
  email: nyzstar@mit.edu
  organization: Dept. of Brain & Cognitive Sci., Massachusetts Inst. of Technol., Cambridge, MA, USA
– sequence: 3
  surname: Jian Dong
  fullname: Jian Dong
  email: a0068947@nus.edu.sg
  organization: Dept. of Electr. & Comput. Eng., Nat. Univ. of Singapore, Singapore, Singapore
– sequence: 4
  surname: Zilei Wang
  fullname: Zilei Wang
  email: zlwang@ustc.edu.cn
  organization: Dept. of Autom., Univ. of Sci. & Technol. of China, Hefei, China
– sequence: 5
  surname: Shuicheng Yan
  fullname: Shuicheng Yan
  email: eleyans@nus.edu.sg
  organization: Dept. of Electr. & Comput. Eng., Nat. Univ. of Singapore, Singapore, Singapore
BackLink http://pascal-francis.inist.fr/vibad/index.php?action=getRecordDetail&idt=26376597$$DView record in Pascal Francis
BookMark eNo9kMFrwjAUh8NwMOd2H-zSy45xL0nTNLsNcSooenDnEtPXEbFpSdrB_vtVFE_vwft-Px7fIxn5xiMhLwymjIF-3282Uw6MTznTeQ7sjoyZThkFUGo07JID1ZzBA3mM8QjAUglqTFa7PrRNdL-YLF1ZoqfbwxFtRxemxo9kXh-wLJ3_SZZ9bXwya-q270znGp84n-yatj-ZkJzhJ3JfmVPE5-uckO-v-X62pOvtYjX7XFPLtegoFykKaa0CVRorRMpLKTgMz6TWQH5gWQWVtFJzRDCaZxZlJqHiuYVUKRATApdeG5oYA1ZFG1xtwl_BoDirKAYVxVlFcVUxRN4ukdZEa05VMN66eMvxTKhMajVwrxfOIeLtnA0tQirxD_Z6Z4Y
CODEN ITMUF8
CitedBy_id crossref_primary_10_3756_artsci_21_233
Cites_doi 10.1145/1276377.1276381
10.1145/1882471.1882475
10.1023/A:1011139631724
10.1561/0600000019
10.1145/1631272.1631359
10.1109/34.1000236
10.1109/CSE.2009.395
10.1007/s11263-007-0090-8
10.1145/1459359.1459418
10.1109/38.946629
10.1145/1015706.1015750
10.1145/1873951.1874028
10.1016/j.patcog.2006.12.019
10.1109/TIP.2009.2038764
10.1109/TKDE.2007.48
10.1023/B:VISI.0000022288.19776.77
10.1145/1460096.1460104
10.1145/966389.966390
10.1007/s11263-006-0031-y
10.1145/1753326.1753643
10.1023/A:1011126920638
10.1145/1526709.1526757
10.1145/1015706.1015720
ContentType Journal Article
Copyright 2014 INIST-CNRS
Copyright_xml – notice: 2014 INIST-CNRS
DBID 97E
RIA
RIE
IQODW
AAYXX
CITATION
DOI 10.1109/TMM.2012.2198801
DatabaseName IEEE All-Society Periodicals Package (ASPP) 2005-present
IEEE All-Society Periodicals Package (ASPP) 1998–Present
IEEE Electronic Library Online
Pascal-Francis
CrossRef
DatabaseTitle CrossRef
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library Online
  url: https://proxy.k.utb.cz/login?url=https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Engineering
Computer Science
Applied Sciences
EISSN 1941-0077
EndPage 1507
ExternalDocumentID 10_1109_TMM_2012_2198801
26376597
6198357
Genre orig-research
GroupedDBID -~X
0R~
29I
4.4
5GY
5VS
6IK
97E
AAJGR
AASAJ
ABQJQ
ABTAH
ABVLG
ACGFO
ACGFS
ACIWK
AENEX
AETIX
AI.
AIBXA
AKJIK
ALLEH
ALMA_UNASSIGNED_HOLDINGS
ATWAV
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CS3
DU5
EBS
EJD
HZ~
H~9
IFIPE
IFJZH
IPLJI
JAVBF
LAI
M43
O9-
OCL
P2P
PQQKQ
RIA
RIE
RNS
TN5
VH1
XFK
ZY4
ABFLS
IQODW
PQEST
AAYXX
AGSQL
CITATION
ID FETCH-LOGICAL-c293t-234e35cc707dac3342d53205074ca08b16f0f5c592ee0a926ce5650f28c047703
IEDL.DBID RIE
ISSN 1520-9210
IngestDate Fri Dec 06 05:51:14 EST 2024
Thu Nov 24 18:30:55 EST 2022
Wed Jun 26 19:20:21 EDT 2024
IsPeerReviewed true
IsScholarly true
Issue 5
Keywords Games with a purpose (GWAP)
Entertainment
Image processing
Algorithmics
Very large databases
Labelling
multimedia computing
Interest
Cheating
Semantics
Database
Learning algorithm
Object location
Multimedia
Computer vision
Refinement method
Image retrieval
Social network
Object recognition
Game theory
Computer games
Annotation
human computing
Artificial intelligence
Indexing
Language English
License CC BY 4.0
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c293t-234e35cc707dac3342d53205074ca08b16f0f5c592ee0a926ce5650f28c047703
PageCount 12
ParticipantIDs crossref_primary_10_1109_TMM_2012_2198801
pascalfrancis_primary_26376597
ieee_primary_6198357
PublicationCentury 2000
PublicationDate 2012-10-01
PublicationDateYYYYMMDD 2012-10-01
PublicationDate_xml – month: 10
  year: 2012
  text: 2012-10-01
  day: 01
PublicationDecade 2010
PublicationPlace New York, NY
PublicationPlace_xml – name: New York, NY
PublicationTitle IEEE transactions on multimedia
PublicationTitleAbbrev TMM
PublicationYear 2012
Publisher IEEE
Institute of Electrical and Electronics Engineers
Publisher_xml – name: IEEE
– name: Institute of Electrical and Electronics Engineers
References ref15
wang (ref24) 2006
ref14
everingham (ref7) 0
ref11
ref10
mori (ref17) 2004
ref1
ref16
ref18
ahn (ref3) 2006
lalonde (ref13) 2007; 26
ostromoukhov (ref19) 2004; 23
ref23
ref26
ref25
ref20
ref22
ref21
ahn (ref2) 2004
ref28
ref27
ref29
ref8
ref9
ref4
ref6
ref5
kang (ref12) 2006
References_xml – start-page: 55
  year: 2006
  ident: ref3
  article-title: Peekaboom: A game for locating objects in images
  publication-title: Proc Conf Human Factors in Computing Systems
  contributor:
    fullname: ahn
– volume: 26
  start-page: 3
  year: 2007
  ident: ref13
  article-title: Photo clip art
  publication-title: ACM Trans Graph
  doi: 10.1145/1276377.1276381
  contributor:
    fullname: lalonde
– start-page: 1719
  year: 2006
  ident: ref12
  article-title: Correlated label propagation with application to multi-label learning
  publication-title: Proc IEEE Computer Society Conf Computer Vision and Pattern Recognition
  contributor:
    fullname: kang
– start-page: 326
  year: 2004
  ident: ref17
  article-title: Recovering human body configurations: Combining segmentation and recognition
  publication-title: Proc IEEE Computer Society Conf Computer Vision and Pattern Recognition
  contributor:
    fullname: mori
– ident: ref9
  doi: 10.1145/1882471.1882475
– ident: ref18
  doi: 10.1023/A:1011139631724
– start-page: 319
  year: 2004
  ident: ref2
  article-title: Labeling images with a computer game
  publication-title: Proc Conf Human Factors in Computing Systems
  contributor:
    fullname: ahn
– ident: ref25
  doi: 10.1561/0600000019
– ident: ref26
  doi: 10.1145/1631272.1631359
– ident: ref6
  doi: 10.1109/34.1000236
– start-page: 647
  year: 2006
  ident: ref24
  article-title: Image annotation refinement using random walk with restarts
  publication-title: Proc ACM Multimedia
  contributor:
    fullname: wang
– ident: ref27
  doi: 10.1109/CSE.2009.395
– year: 0
  ident: ref7
  publication-title: The PASCAL Visual Object Classes Challenge 2007 (VOC2007) Results
  contributor:
    fullname: everingham
– ident: ref22
  doi: 10.1007/s11263-007-0090-8
– ident: ref16
  doi: 10.1145/1459359.1459418
– ident: ref20
  doi: 10.1109/38.946629
– volume: 23
  start-page: 488
  year: 2004
  ident: ref19
  article-title: Fast hierarchical importance sampling with blue noise properties
  publication-title: ACM Trans Graph
  doi: 10.1145/1015706.1015750
  contributor:
    fullname: ostromoukhov
– ident: ref29
  doi: 10.1145/1873951.1874028
– ident: ref28
  doi: 10.1016/j.patcog.2006.12.019
– ident: ref4
  doi: 10.1109/TIP.2009.2038764
– ident: ref5
  doi: 10.1109/TKDE.2007.48
– ident: ref8
  doi: 10.1023/B:VISI.0000022288.19776.77
– ident: ref11
  doi: 10.1145/1460096.1460104
– ident: ref1
  doi: 10.1145/966389.966390
– ident: ref10
  doi: 10.1007/s11263-006-0031-y
– ident: ref23
  doi: 10.1145/1753326.1753643
– ident: ref14
  doi: 10.1023/A:1011126920638
– ident: ref15
  doi: 10.1145/1526709.1526757
– ident: ref21
  doi: 10.1145/1015706.1015720
SSID ssj0014507
Score 2.0761175
Snippet Having sufficient training images with fully annotated object locations is undoubtedly critical for modern learning-based image annotation, retrieval, and...
SourceID crossref
pascalfrancis
ieee
SourceType Aggregation Database
Index Database
Publisher
StartPage 1496
SubjectTerms Accuracy
Applied sciences
Artificial intelligence
Computer science; control theory; systems
Computer systems and distributed systems. User interface
Exact sciences and technology
Games
Games with a purpose (GWAP)
human computing
Humans
image processing
Information systems. Data bases
Labeling
Memory organisation. Data processing
Multimedia communication
multimedia computing
Noise measurement
Pattern recognition. Digital image processing. Computational geometry
Semantics
Software
Title Purposive Hidden-Object-Game: Embedding Human Computation in Popular Game
URI https://ieeexplore.ieee.org/document/6198357
Volume 14
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1LT8MwDLYGJzjwRoyXcuCCREaWpk3LDSHGQBpwAIlblbquhBAdgu3Cr8dJuwkQB26VmkSRPye2E8cfwBFpyvrGWamjuJQmxUKmxpGMMEFriCoMZBOj22T4aG6e4qcOnMzfwhBRSD6jnv8Md_nlGKf-qOyUnX12GOwCLNgsad5qzW8MTByeRrM5UjLjOGZ2Jamy04fRyOdw6R6vTlbX_g8TFDhVfEak-2ChVA2bxTcTM1iF0WxyTWbJS286KXr4-atu439nvwYrra8pzhvlWIcO1RuwOuNxEO2y3oDlb0UJN-H6nmU_9lntYujri9TyrvCHNfLKvdKZuHwtqPQGT4Tzf9EMF_AVz7W4D4Rg78I33oLHweXDxVC2hAsS2epPGC9DUYxolS0dRpHRpeeNYMkadCot-kmlqhjjTBMpl-kEif1BVekUlbG8d2zDYj2uaQdE5XRSpmWh-8g-gtWFRm19sUJUkdOZ6cLxDIP8ramrkYd4RGU545V7vPIWry5semnO27WC7MLhD9Dm_3XCuyYHSrt_99uDJT96k5G3D4uT9ykdsGcxKQ6DSn0B2_fJQQ
link.rule.ids 314,780,784,796,27924,27925,54758
linkProvider IEEE
linkToHtml http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1NTxsxEB2l6YH20JRQ1LRAfeCChIPj9X71hhCQFDZwCBK3lXd2VkKIDYLk0l_fsXcTAeqB20rr_dA82_PsGc8D2CdN6cjYWOogLKVJsJCJsSQDjDA2RBV6sYlsGo1vzJ_b8LYDh-uzMETkk89o6C59LL-c49JtlR0x2WfCEH-Aj6Fhntuc1lrHDEzoD0ezQ1Iy5ZXMKiip0qNZlrksLj3k8ckddvTKCXlVFZcTaZ_ZLFWjZ_HCyZz1IFv9XpNbcj9cLooh_n1TufG9__8VvrRsUxw33WMTOlT3obdSchDtwO7D5xdlCbdgcs3Wn7u8djF2FUZqeVW47Rp5bh_otzh9KKh0Lk_4CIBoXucRFne1uPaSYE_CNf4GN2ens5OxbCUXJLLfXzBihoIQMVZxaTEIjC6dcgRb1qBVSTGKKlWFGKaaSNlUR0jMCFWlE1Qm5tljG7r1vKbvICqrozIpCz1CZgmxLjTq2JUrRBVYnZoBHKwwyB-byhq5X5GoNGe8codX3uI1gC1nzXW71pAD2HsF2vq-jnje5KXSj_8_9ws2xrPsMr-cTC9-wif3pSY_bwe6i6cl7TLPWBR7vnv9A6LyzJQ
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Purposive+Hidden-Object-Game%3A+Embedding+Human+Computation+in+Popular+Game&rft.jtitle=IEEE+transactions+on+multimedia&rft.au=Jiashi+Feng&rft.au=Yuzhao+Ni&rft.au=Jian+Dong&rft.au=Zilei+Wang&rft.date=2012-10-01&rft.pub=IEEE&rft.issn=1520-9210&rft.eissn=1941-0077&rft.volume=14&rft.issue=5&rft.spage=1496&rft.epage=1507&rft_id=info:doi/10.1109%2FTMM.2012.2198801&rft.externalDocID=6198357
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1520-9210&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1520-9210&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1520-9210&client=summon