Purposive Hidden-Object-Game: Embedding Human Computation in Popular Game
Having sufficient training images with fully annotated object locations is undoubtedly critical for modern learning-based image annotation, retrieval, and object detection methods. Typically, collecting such annotations for large-scale datasets is notoriously tedious because the process involves amo...
Saved in:
Published in | IEEE transactions on multimedia Vol. 14; no. 5; pp. 1496 - 1507 |
---|---|
Main Authors | , , , , |
Format | Journal Article |
Language | English |
Published |
New York, NY
IEEE
01.10.2012
Institute of Electrical and Electronics Engineers |
Subjects | |
Online Access | Get full text |
Cover
Loading…
Abstract | Having sufficient training images with fully annotated object locations is undoubtedly critical for modern learning-based image annotation, retrieval, and object detection methods. Typically, collecting such annotations for large-scale datasets is notoriously tedious because the process involves amount of manual cropping and hand labeling operations. In this work, following the principle of games with a purpose (GWAP), we design a so-called purposive hidden-object-game (P-HOG), which imperceptibly embeds localizing objects into enjoyable playing game process and thus attracts many people to make voluntary contribution to annotating images. In particular, besides preserving the interestingness as popular HOG games, P-HOG is able to automatically generate satisfactory game images (i.e., "hide" certain items into target images) by integrating several semantic and visual processing techniques. P-HOG is also built in an effective mechanism to prevent the players from cheating. The mechanism inherits the merit of Recaptcha and identifies potential cheating behavior based on the annotation accuracy of some known items. Moreover, P-HOG will filter noisy annotations effectively based on a weighted majority method and improve the accuracy of the raw annotations from the players. Most importantly, players only play P-HOG for entertainment purpose and they are unaware of the background data collection procedure. The collected data are used towards constructing a large database, which may benefit general learning-based algorithms for multimedia tasks. To the best of our knowledge, this is the first work dedicated to such a specific and important task under the GWAP framework. We conduct a pilot study of the game prototype and the comprehensive experiments show that the P-HOG appeals to general players, and is effective for collecting massive object locations with satisfactory accuracy, which further boosts the algorithmic performances for both tag refinement and image annotation tasks. |
---|---|
AbstractList | Having sufficient training images with fully annotated object locations is undoubtedly critical for modern learning-based image annotation, retrieval, and object detection methods. Typically, collecting such annotations for large-scale datasets is notoriously tedious because the process involves amount of manual cropping and hand labeling operations. In this work, following the principle of games with a purpose (GWAP), we design a so-called purposive hidden-object-game (P-HOG), which imperceptibly embeds localizing objects into enjoyable playing game process and thus attracts many people to make voluntary contribution to annotating images. In particular, besides preserving the interestingness as popular HOG games, P-HOG is able to automatically generate satisfactory game images (i.e., "hide" certain items into target images) by integrating several semantic and visual processing techniques. P-HOG is also built in an effective mechanism to prevent the players from cheating. The mechanism inherits the merit of Recaptcha and identifies potential cheating behavior based on the annotation accuracy of some known items. Moreover, P-HOG will filter noisy annotations effectively based on a weighted majority method and improve the accuracy of the raw annotations from the players. Most importantly, players only play P-HOG for entertainment purpose and they are unaware of the background data collection procedure. The collected data are used towards constructing a large database, which may benefit general learning-based algorithms for multimedia tasks. To the best of our knowledge, this is the first work dedicated to such a specific and important task under the GWAP framework. We conduct a pilot study of the game prototype and the comprehensive experiments show that the P-HOG appeals to general players, and is effective for collecting massive object locations with satisfactory accuracy, which further boosts the algorithmic performances for both tag refinement and image annotation tasks. |
Author | Jian Dong Zilei Wang Shuicheng Yan Yuzhao Ni Jiashi Feng |
Author_xml | – sequence: 1 surname: Jiashi Feng fullname: Jiashi Feng email: a0066331@nus.edu.sg organization: Dept. of Electr. & Comput. Eng., Nat. Univ. of Singapore, Singapore, Singapore – sequence: 2 surname: Yuzhao Ni fullname: Yuzhao Ni email: nyzstar@mit.edu organization: Dept. of Brain & Cognitive Sci., Massachusetts Inst. of Technol., Cambridge, MA, USA – sequence: 3 surname: Jian Dong fullname: Jian Dong email: a0068947@nus.edu.sg organization: Dept. of Electr. & Comput. Eng., Nat. Univ. of Singapore, Singapore, Singapore – sequence: 4 surname: Zilei Wang fullname: Zilei Wang email: zlwang@ustc.edu.cn organization: Dept. of Autom., Univ. of Sci. & Technol. of China, Hefei, China – sequence: 5 surname: Shuicheng Yan fullname: Shuicheng Yan email: eleyans@nus.edu.sg organization: Dept. of Electr. & Comput. Eng., Nat. Univ. of Singapore, Singapore, Singapore |
BackLink | http://pascal-francis.inist.fr/vibad/index.php?action=getRecordDetail&idt=26376597$$DView record in Pascal Francis |
BookMark | eNo9kMFrwjAUh8NwMOd2H-zSy45xL0nTNLsNcSooenDnEtPXEbFpSdrB_vtVFE_vwft-Px7fIxn5xiMhLwymjIF-3282Uw6MTznTeQ7sjoyZThkFUGo07JID1ZzBA3mM8QjAUglqTFa7PrRNdL-YLF1ZoqfbwxFtRxemxo9kXh-wLJ3_SZZ9bXwya-q270znGp84n-yatj-ZkJzhJ3JfmVPE5-uckO-v-X62pOvtYjX7XFPLtegoFykKaa0CVRorRMpLKTgMz6TWQH5gWQWVtFJzRDCaZxZlJqHiuYVUKRATApdeG5oYA1ZFG1xtwl_BoDirKAYVxVlFcVUxRN4ukdZEa05VMN66eMvxTKhMajVwrxfOIeLtnA0tQirxD_Z6Z4Y |
CODEN | ITMUF8 |
CitedBy_id | crossref_primary_10_3756_artsci_21_233 |
Cites_doi | 10.1145/1276377.1276381 10.1145/1882471.1882475 10.1023/A:1011139631724 10.1561/0600000019 10.1145/1631272.1631359 10.1109/34.1000236 10.1109/CSE.2009.395 10.1007/s11263-007-0090-8 10.1145/1459359.1459418 10.1109/38.946629 10.1145/1015706.1015750 10.1145/1873951.1874028 10.1016/j.patcog.2006.12.019 10.1109/TIP.2009.2038764 10.1109/TKDE.2007.48 10.1023/B:VISI.0000022288.19776.77 10.1145/1460096.1460104 10.1145/966389.966390 10.1007/s11263-006-0031-y 10.1145/1753326.1753643 10.1023/A:1011126920638 10.1145/1526709.1526757 10.1145/1015706.1015720 |
ContentType | Journal Article |
Copyright | 2014 INIST-CNRS |
Copyright_xml | – notice: 2014 INIST-CNRS |
DBID | 97E RIA RIE IQODW AAYXX CITATION |
DOI | 10.1109/TMM.2012.2198801 |
DatabaseName | IEEE All-Society Periodicals Package (ASPP) 2005-present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library Online Pascal-Francis CrossRef |
DatabaseTitle | CrossRef |
DatabaseTitleList | |
Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library Online url: https://proxy.k.utb.cz/login?url=https://ieeexplore.ieee.org/ sourceTypes: Publisher |
DeliveryMethod | fulltext_linktorsrc |
Discipline | Engineering Computer Science Applied Sciences |
EISSN | 1941-0077 |
EndPage | 1507 |
ExternalDocumentID | 10_1109_TMM_2012_2198801 26376597 6198357 |
Genre | orig-research |
GroupedDBID | -~X 0R~ 29I 4.4 5GY 5VS 6IK 97E AAJGR AASAJ ABQJQ ABTAH ABVLG ACGFO ACGFS ACIWK AENEX AETIX AI. AIBXA AKJIK ALLEH ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD HZ~ H~9 IFIPE IFJZH IPLJI JAVBF LAI M43 O9- OCL P2P PQQKQ RIA RIE RNS TN5 VH1 XFK ZY4 ABFLS IQODW PQEST AAYXX AGSQL CITATION |
ID | FETCH-LOGICAL-c293t-234e35cc707dac3342d53205074ca08b16f0f5c592ee0a926ce5650f28c047703 |
IEDL.DBID | RIE |
ISSN | 1520-9210 |
IngestDate | Fri Dec 06 05:51:14 EST 2024 Thu Nov 24 18:30:55 EST 2022 Wed Jun 26 19:20:21 EDT 2024 |
IsPeerReviewed | true |
IsScholarly | true |
Issue | 5 |
Keywords | Games with a purpose (GWAP) Entertainment Image processing Algorithmics Very large databases Labelling multimedia computing Interest Cheating Semantics Database Learning algorithm Object location Multimedia Computer vision Refinement method Image retrieval Social network Object recognition Game theory Computer games Annotation human computing Artificial intelligence Indexing |
Language | English |
License | CC BY 4.0 |
LinkModel | DirectLink |
MergedId | FETCHMERGED-LOGICAL-c293t-234e35cc707dac3342d53205074ca08b16f0f5c592ee0a926ce5650f28c047703 |
PageCount | 12 |
ParticipantIDs | crossref_primary_10_1109_TMM_2012_2198801 pascalfrancis_primary_26376597 ieee_primary_6198357 |
PublicationCentury | 2000 |
PublicationDate | 2012-10-01 |
PublicationDateYYYYMMDD | 2012-10-01 |
PublicationDate_xml | – month: 10 year: 2012 text: 2012-10-01 day: 01 |
PublicationDecade | 2010 |
PublicationPlace | New York, NY |
PublicationPlace_xml | – name: New York, NY |
PublicationTitle | IEEE transactions on multimedia |
PublicationTitleAbbrev | TMM |
PublicationYear | 2012 |
Publisher | IEEE Institute of Electrical and Electronics Engineers |
Publisher_xml | – name: IEEE – name: Institute of Electrical and Electronics Engineers |
References | ref15 wang (ref24) 2006 ref14 everingham (ref7) 0 ref11 ref10 mori (ref17) 2004 ref1 ref16 ref18 ahn (ref3) 2006 lalonde (ref13) 2007; 26 ostromoukhov (ref19) 2004; 23 ref23 ref26 ref25 ref20 ref22 ref21 ahn (ref2) 2004 ref28 ref27 ref29 ref8 ref9 ref4 ref6 ref5 kang (ref12) 2006 |
References_xml | – start-page: 55 year: 2006 ident: ref3 article-title: Peekaboom: A game for locating objects in images publication-title: Proc Conf Human Factors in Computing Systems contributor: fullname: ahn – volume: 26 start-page: 3 year: 2007 ident: ref13 article-title: Photo clip art publication-title: ACM Trans Graph doi: 10.1145/1276377.1276381 contributor: fullname: lalonde – start-page: 1719 year: 2006 ident: ref12 article-title: Correlated label propagation with application to multi-label learning publication-title: Proc IEEE Computer Society Conf Computer Vision and Pattern Recognition contributor: fullname: kang – start-page: 326 year: 2004 ident: ref17 article-title: Recovering human body configurations: Combining segmentation and recognition publication-title: Proc IEEE Computer Society Conf Computer Vision and Pattern Recognition contributor: fullname: mori – ident: ref9 doi: 10.1145/1882471.1882475 – ident: ref18 doi: 10.1023/A:1011139631724 – start-page: 319 year: 2004 ident: ref2 article-title: Labeling images with a computer game publication-title: Proc Conf Human Factors in Computing Systems contributor: fullname: ahn – ident: ref25 doi: 10.1561/0600000019 – ident: ref26 doi: 10.1145/1631272.1631359 – ident: ref6 doi: 10.1109/34.1000236 – start-page: 647 year: 2006 ident: ref24 article-title: Image annotation refinement using random walk with restarts publication-title: Proc ACM Multimedia contributor: fullname: wang – ident: ref27 doi: 10.1109/CSE.2009.395 – year: 0 ident: ref7 publication-title: The PASCAL Visual Object Classes Challenge 2007 (VOC2007) Results contributor: fullname: everingham – ident: ref22 doi: 10.1007/s11263-007-0090-8 – ident: ref16 doi: 10.1145/1459359.1459418 – ident: ref20 doi: 10.1109/38.946629 – volume: 23 start-page: 488 year: 2004 ident: ref19 article-title: Fast hierarchical importance sampling with blue noise properties publication-title: ACM Trans Graph doi: 10.1145/1015706.1015750 contributor: fullname: ostromoukhov – ident: ref29 doi: 10.1145/1873951.1874028 – ident: ref28 doi: 10.1016/j.patcog.2006.12.019 – ident: ref4 doi: 10.1109/TIP.2009.2038764 – ident: ref5 doi: 10.1109/TKDE.2007.48 – ident: ref8 doi: 10.1023/B:VISI.0000022288.19776.77 – ident: ref11 doi: 10.1145/1460096.1460104 – ident: ref1 doi: 10.1145/966389.966390 – ident: ref10 doi: 10.1007/s11263-006-0031-y – ident: ref23 doi: 10.1145/1753326.1753643 – ident: ref14 doi: 10.1023/A:1011126920638 – ident: ref15 doi: 10.1145/1526709.1526757 – ident: ref21 doi: 10.1145/1015706.1015720 |
SSID | ssj0014507 |
Score | 2.0761175 |
Snippet | Having sufficient training images with fully annotated object locations is undoubtedly critical for modern learning-based image annotation, retrieval, and... |
SourceID | crossref pascalfrancis ieee |
SourceType | Aggregation Database Index Database Publisher |
StartPage | 1496 |
SubjectTerms | Accuracy Applied sciences Artificial intelligence Computer science; control theory; systems Computer systems and distributed systems. User interface Exact sciences and technology Games Games with a purpose (GWAP) human computing Humans image processing Information systems. Data bases Labeling Memory organisation. Data processing Multimedia communication multimedia computing Noise measurement Pattern recognition. Digital image processing. Computational geometry Semantics Software |
Title | Purposive Hidden-Object-Game: Embedding Human Computation in Popular Game |
URI | https://ieeexplore.ieee.org/document/6198357 |
Volume | 14 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1LT8MwDLYGJzjwRoyXcuCCREaWpk3LDSHGQBpwAIlblbquhBAdgu3Cr8dJuwkQB26VmkSRPye2E8cfwBFpyvrGWamjuJQmxUKmxpGMMEFriCoMZBOj22T4aG6e4qcOnMzfwhBRSD6jnv8Md_nlGKf-qOyUnX12GOwCLNgsad5qzW8MTByeRrM5UjLjOGZ2Jamy04fRyOdw6R6vTlbX_g8TFDhVfEak-2ChVA2bxTcTM1iF0WxyTWbJS286KXr4-atu439nvwYrra8pzhvlWIcO1RuwOuNxEO2y3oDlb0UJN-H6nmU_9lntYujri9TyrvCHNfLKvdKZuHwtqPQGT4Tzf9EMF_AVz7W4D4Rg78I33oLHweXDxVC2hAsS2epPGC9DUYxolS0dRpHRpeeNYMkadCot-kmlqhjjTBMpl-kEif1BVekUlbG8d2zDYj2uaQdE5XRSpmWh-8g-gtWFRm19sUJUkdOZ6cLxDIP8ramrkYd4RGU545V7vPIWry5semnO27WC7MLhD9Dm_3XCuyYHSrt_99uDJT96k5G3D4uT9ykdsGcxKQ6DSn0B2_fJQQ |
link.rule.ids | 314,780,784,796,27924,27925,54758 |
linkProvider | IEEE |
linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1NTxsxEB2l6YH20JRQ1LRAfeCChIPj9X71hhCQFDZwCBK3lXd2VkKIDYLk0l_fsXcTAeqB20rr_dA82_PsGc8D2CdN6cjYWOogLKVJsJCJsSQDjDA2RBV6sYlsGo1vzJ_b8LYDh-uzMETkk89o6C59LL-c49JtlR0x2WfCEH-Aj6Fhntuc1lrHDEzoD0ezQ1Iy5ZXMKiip0qNZlrksLj3k8ckddvTKCXlVFZcTaZ_ZLFWjZ_HCyZz1IFv9XpNbcj9cLooh_n1TufG9__8VvrRsUxw33WMTOlT3obdSchDtwO7D5xdlCbdgcs3Wn7u8djF2FUZqeVW47Rp5bh_otzh9KKh0Lk_4CIBoXucRFne1uPaSYE_CNf4GN2ens5OxbCUXJLLfXzBihoIQMVZxaTEIjC6dcgRb1qBVSTGKKlWFGKaaSNlUR0jMCFWlE1Qm5tljG7r1vKbvICqrozIpCz1CZgmxLjTq2JUrRBVYnZoBHKwwyB-byhq5X5GoNGe8codX3uI1gC1nzXW71pAD2HsF2vq-jnje5KXSj_8_9ws2xrPsMr-cTC9-wif3pSY_bwe6i6cl7TLPWBR7vnv9A6LyzJQ |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Purposive+Hidden-Object-Game%3A+Embedding+Human+Computation+in+Popular+Game&rft.jtitle=IEEE+transactions+on+multimedia&rft.au=Jiashi+Feng&rft.au=Yuzhao+Ni&rft.au=Jian+Dong&rft.au=Zilei+Wang&rft.date=2012-10-01&rft.pub=IEEE&rft.issn=1520-9210&rft.eissn=1941-0077&rft.volume=14&rft.issue=5&rft.spage=1496&rft.epage=1507&rft_id=info:doi/10.1109%2FTMM.2012.2198801&rft.externalDocID=6198357 |
thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1520-9210&client=summon |
thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1520-9210&client=summon |
thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1520-9210&client=summon |