Absolute and Relative User Perception of Classification Accuracy in an Affective Video Game
Abstract Classification algorithms are used in affective computing to classify the state of the user and adapt the computer's behaviour, but it is unclear how classification accuracy influences the overall user experience. We present a study in which classification accuracy is artificially pre-...
Saved in:
| Published in | Interacting with computers Vol. 29; no. 2; pp. 271 - 286 |
|---|---|
| Main Authors | , , |
| Format | Journal Article |
| Language | English |
| Published |
Oxford University Press
01.03.2017
|
| Subjects | |
| Online Access | Get full text |
Cover
Loading…
| Abstract | Abstract
Classification algorithms are used in affective computing to classify the state of the user and adapt the computer's behaviour, but it is unclear how classification accuracy influences the overall user experience. We present a study in which classification accuracy is artificially pre-defined and used to adapt to the difficulty of a video game. Eighty subjects played the game and were told that difficulty would be adapted according to the measured brain activity. They played the game twice, with different classification accuracies, and then reported different aspects of their overall game experience using questionnaires. Classification accuracy was correlated with both in-game fun (r = 0.46) and satisfaction with the difficulty adaptation (r = 0.56). Most subjects could perceive a difference between two classification accuracies that differed by 16.7%. We tentatively posit that, for affective video games, an acceptable classification accuracy is 70–80%. Furthermore, studies that attempt to improve affect classification accuracy should aim for a practically meaningful improvement of 10%. |
|---|---|
| AbstractList | Abstract
Classification algorithms are used in affective computing to classify the state of the user and adapt the computer's behaviour, but it is unclear how classification accuracy influences the overall user experience. We present a study in which classification accuracy is artificially pre-defined and used to adapt to the difficulty of a video game. Eighty subjects played the game and were told that difficulty would be adapted according to the measured brain activity. They played the game twice, with different classification accuracies, and then reported different aspects of their overall game experience using questionnaires. Classification accuracy was correlated with both in-game fun (r = 0.46) and satisfaction with the difficulty adaptation (r = 0.56). Most subjects could perceive a difference between two classification accuracies that differed by 16.7%. We tentatively posit that, for affective video games, an acceptable classification accuracy is 70–80%. Furthermore, studies that attempt to improve affect classification accuracy should aim for a practically meaningful improvement of 10%. |
| Author | McCrea, Sean Geršak, Gregor Novak, Domen |
| Author_xml | – sequence: 1 givenname: Sean surname: McCrea fullname: McCrea, Sean organization: 1 University of Wyoming, 1000 E University Avenue, Laramie, WY 82071, USA – sequence: 2 givenname: Gregor surname: Geršak fullname: Geršak, Gregor organization: 2 Faculty of Electrical Engineering, University of Ljubljana, Trzaska cesta 25, SI-1000 Ljubljana, Slovenia – sequence: 3 givenname: Domen surname: Novak fullname: Novak, Domen email: dnovak1@uwyo.edu organization: 1 University of Wyoming, 1000 E University Avenue, Laramie, WY 82071, USA |
| BookMark | eNp9kEFLwzAUx4NMcJte_AS5eBGqSZM2ybEMncJAEefFQ0nSF4h0zUg6x769dfUk4uHxh8fv93j8Z2jShQ4QuqTkhhLFbv3eDrMneXmCplQKlglV0AmaElWwrOBMnqFZSh-EECEkn6L3yqTQ7nrAumvwC7S695-A1wkifoZoYdv70OHg8KLVKXnnrT5uKmt3UdsD9t2g4so5sEf1zTcQ8FJv4BydOt0muPjJOVrf370uHrLV0_JxUa0ymyvRZ05q6gTjhhibc10YC43QhjrJlKBGgnK6aErDhFGkaFxeSs654MwYk1vSsDki410bQ0oRXG19f_yyj9q3NSX1dzn1UE49ljMo17-UbfQbHQ9_w1cjHHbb_7gvgLR3ow |
| CitedBy_id | crossref_primary_10_1109_TAFFC_2023_3276425 crossref_primary_10_1186_s12984_023_01163_2 crossref_primary_10_2196_25771 crossref_primary_10_3389_fnrgo_2020_602504 crossref_primary_10_3389_fnins_2019_01278 crossref_primary_10_1109_TAFFC_2023_3236265 |
| Cites_doi | 10.1109/34.954607 10.1145/1232743.1232769 10.1177/1745691610393521 10.1016/0005-7916(94)90063-9 10.1016/S0092-6566(03)00046-1 10.1109/TSMCA.2011.2164242 10.4017/gt.2015.14.1.001.00 10.1109/TCYB.2013.2282279 10.1016/S0002-8703(02)94797-6 10.1109/TAFFC.2014.2326870 10.1109/GEM.2014.7048094 10.1109/T-AFFC.2012.33 10.1518/hfes.45.4.635.27088 10.1016/j.neunet.2009.05.005 10.1186/1743-0003-11-64 10.1109/T-AFFC.2012.28 10.1016/j.ijhcs.2006.11.011 10.1145/2702123.2702454 10.1016/j.ijhcs.2007.10.011 10.1080/00222895.2015.1035430 10.1016/j.intcom.2008.10.011 10.1109/TSMCA.2011.2116000 10.1088/1741-2560/8/2/025005 10.1109/T-AFFC.2011.7 10.1016/j.jrp.2005.08.007 10.1109/TNSRE.2011.2160357 10.1016/j.math.2011.05.006 10.1109/TAFFC.2014.2326402 10.1016/j.biopsycho.2010.03.010 10.1080/02701367.1989.10607413 10.1080/10447310902963944 10.1109/TRO.2008.2001362 10.1111/j.1469-8986.2009.00947.x 10.1016/0301-0511(95)05116-3 10.1016/j.intcom.2012.04.003 10.1016/j.cmpb.2013.02.003 |
| ContentType | Journal Article |
| Copyright | The Author 2016. Published by Oxford University Press on behalf of The British Computer Society. All rights reserved. For Permissions, please email: journals.permissions@oup.com 2016 |
| Copyright_xml | – notice: The Author 2016. Published by Oxford University Press on behalf of The British Computer Society. All rights reserved. For Permissions, please email: journals.permissions@oup.com 2016 |
| DBID | AAYXX CITATION |
| DOI | 10.1093/iwc/iww026 |
| DatabaseName | CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISSN | 1873-7951 |
| EndPage | 286 |
| ExternalDocumentID | 10_1093_iwc_iww026 10.1093/iwc/iww026 |
| GroupedDBID | .2P .DC .I3 0R~ 1TH 4.4 48X 5GY 5WD 71M 77K AABZA AACTN AACZT AAFXQ AAIJN AAJKP AAJQQ AAMVS AAMZS AAOGV AAPQZ AAPXW AARHZ AASNB AAUAY AAUOS AAUQX AAVAP ABBGM ABDTM ABEUO ABICN ABIVO ABIXL ABJNI ABKEB ABNHQ ABNKS ABPTD ABQLI ABWST ABXVK ABXVV ABZBJ ACDXO ACGFS ACHQT ACMCV ACUFI ACUTJ ACUTO ACVJI ACYTK ADBBV ADBKU ADEYI ADGZP ADHKW ADHZD ADIPN ADJQC ADLMC ADLOL ADOCK ADQBN ADQIT ADRDM ADRIX ADRTK ADVEK ADYVW ADZXQ AECKG AEGPL AEJER AEJOX AEKKA AEKSI AEMDU AENEX AENZO AEPUE AETBJ AEWNT AFFZL AFHLB AFIYH AFOFC AFVIK AFVSF AFXEN AGFCL AGINJ AGQXC AGSYK AHXPO AIDGQ AJECE AJEEA AJEUX ALMA_UNASSIGNED_HOLDINGS ALTZX ALUQC AOLPF APIBT ASPYK ATGXG AVWKF AXUDD AYLYT AZVOD BAYMD BCRHZ BEFXN BEYMZ BFFAM BFGLT BGNUA BHONS BHZBG BKEBE BMSTW BOXDG BPEOZ BQUQU BTQHN BTRTY BVRKM BZYEK CDBKE CS3 DAKXR DDUBX DILTD DU5 D~K EBS EE~ EJD ETYVG EYXSX F5P F9B FDB FLIZI FLUFQ FOEOM FOTVD FQBLK G-Q GAOTZ GAUVT GJXCC H5~ HAR HW0 HZ~ IHE J21 JAVBF KBUDW KOP KSI KSN MJWOD ML0 MXSPP N9A NGC NMDNZ NOMLY NOYVH NQ- NVLIB O9- OAIJC OAWHX OCL OCZFY ODMLO OJQWA OJZSN OKKKP OPAEJ OWPYF OXVUA OZT O~Y P2P PAFKI PEELM PLIXB Q1. Q5Y RIG ROX ROZ RUSNO RW1 RXO SDF TJH TJJ TJP TJX YADRA YAJVU YAYTL YKOAZ YXANX ~S- ~SN AAYXX ABAZT ABDFA ABEJV ABGNP ABPQP ABVGC ABVLG ACOZV ACUXJ ADAES ADMLS ADNBA ADYJX ADYLA AFIQY AGORE AHMMS AJBYB AJNCP ALXQX ANAKG CITATION JXSIZ |
| ID | FETCH-LOGICAL-c297t-f8a1f734b0bc24a5bced7ab1f83971b8e9fa5d6b37b905df268444743bbb2c0d3 |
| ISSN | 0953-5438 |
| IngestDate | Thu Apr 24 23:02:24 EDT 2025 Tue Jul 01 02:03:48 EDT 2025 Wed Sep 11 04:48:45 EDT 2024 |
| IsDoiOpenAccess | false |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 2 |
| Keywords | Affective computing physiological computing machine learning user experience computer games |
| Language | English |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-c297t-f8a1f734b0bc24a5bced7ab1f83971b8e9fa5d6b37b905df268444743bbb2c0d3 |
| OpenAccessLink | https://academic.oup.com/iwc/article-pdf/29/2/271/10296479/iww026.pdf |
| PageCount | 16 |
| ParticipantIDs | crossref_citationtrail_10_1093_iwc_iww026 crossref_primary_10_1093_iwc_iww026 oup_primary_10_1093_iwc_iww026 |
| ProviderPackageCode | CITATION AAYXX |
| PublicationCentury | 2000 |
| PublicationDate | 2017-03-01 |
| PublicationDateYYYYMMDD | 2017-03-01 |
| PublicationDate_xml | – month: 03 year: 2017 text: 2017-03-01 day: 01 |
| PublicationDecade | 2010 |
| PublicationTitle | Interacting with computers |
| PublicationYear | 2017 |
| Publisher | Oxford University Press |
| Publisher_xml | – name: Oxford University Press |
| References | Picot (2016071123264944000_iww026v1.33) 2012; 42 2016071123264944000_iww026v1.29 da Silva Cameirao (2016071123264944000_iww026v1.11) 2011; 29 2016071123264944000_iww026v1.25 Shen (2016071123264944000_iww026v1.37) 2009; 12 Nagle (2016071123264944000_iww026v1.26) 2015; 14 2016071123264944000_iww026v1.27 2016071123264944000_iww026v1.28 2016071123264944000_iww026v1.21 2016071123264944000_iww026v1.22 2016071123264944000_iww026v1.23 2016071123264944000_iww026v1.24 2016071123264944000_iww026v1.40 2016071123264944000_iww026v1.41 2016071123264944000_iww026v1.20 2016071123264944000_iww026v1.42 2016071123264944000_iww026v1.2 2016071123264944000_iww026v1.1 2016071123264944000_iww026v1.4 2016071123264944000_iww026v1.3 2016071123264944000_iww026v1.6 2016071123264944000_iww026v1.18 2016071123264944000_iww026v1.5 2016071123264944000_iww026v1.19 2016071123264944000_iww026v1.8 2016071123264944000_iww026v1.7 2016071123264944000_iww026v1.14 2016071123264944000_iww026v1.36 2016071123264944000_iww026v1.9 2016071123264944000_iww026v1.15 2016071123264944000_iww026v1.16 2016071123264944000_iww026v1.38 2016071123264944000_iww026v1.17 2016071123264944000_iww026v1.39 2016071123264944000_iww026v1.10 2016071123264944000_iww026v1.32 2016071123264944000_iww026v1.12 2016071123264944000_iww026v1.34 2016071123264944000_iww026v1.13 2016071123264944000_iww026v1.35 2016071123264944000_iww026v1.30 2016071123264944000_iww026v1.31 |
| References_xml | – ident: 2016071123264944000_iww026v1.32 doi: 10.1109/34.954607 – ident: 2016071123264944000_iww026v1.8 doi: 10.1145/1232743.1232769 – ident: 2016071123264944000_iww026v1.16 doi: 10.1177/1745691610393521 – ident: 2016071123264944000_iww026v1.4 doi: 10.1016/0005-7916(94)90063-9 – ident: 2016071123264944000_iww026v1.18 doi: 10.1016/S0092-6566(03)00046-1 – volume: 42 start-page: 764 year: 2012 ident: 2016071123264944000_iww026v1.33 article-title: On-line detection of drowsiness using brain and visual information publication-title: IEEE Trans. Syst. Man Cybern. A Syst. Hum. doi: 10.1109/TSMCA.2011.2164242 – volume: 14 start-page: 32 year: 2015 ident: 2016071123264944000_iww026v1.26 article-title: Increased enjoyment using a tablet-based serious game with regularly changing visual elements: a pilot study publication-title: Gerontechnology doi: 10.4017/gt.2015.14.1.001.00 – ident: 2016071123264944000_iww026v1.39 doi: 10.1109/TCYB.2013.2282279 – ident: 2016071123264944000_iww026v1.20 doi: 10.1016/S0002-8703(02)94797-6 – ident: 2016071123264944000_iww026v1.30 doi: 10.1109/TAFFC.2014.2326870 – ident: 2016071123264944000_iww026v1.27 doi: 10.1109/GEM.2014.7048094 – ident: 2016071123264944000_iww026v1.36 doi: 10.1109/T-AFFC.2012.33 – ident: 2016071123264944000_iww026v1.41 doi: 10.1518/hfes.45.4.635.27088 – ident: 2016071123264944000_iww026v1.7 doi: 10.1016/j.neunet.2009.05.005 – ident: 2016071123264944000_iww026v1.31 doi: 10.1186/1743-0003-11-64 – ident: 2016071123264944000_iww026v1.40 doi: 10.1109/T-AFFC.2012.28 – ident: 2016071123264944000_iww026v1.24 doi: 10.1016/j.ijhcs.2006.11.011 – ident: 2016071123264944000_iww026v1.13 doi: 10.1145/2702123.2702454 – ident: 2016071123264944000_iww026v1.2 doi: 10.1016/j.ijhcs.2007.10.011 – ident: 2016071123264944000_iww026v1.38 doi: 10.1080/00222895.2015.1035430 – ident: 2016071123264944000_iww026v1.12 doi: 10.1016/j.intcom.2008.10.011 – ident: 2016071123264944000_iww026v1.10 – ident: 2016071123264944000_iww026v1.6 doi: 10.1109/TSMCA.2011.2116000 – ident: 2016071123264944000_iww026v1.42 doi: 10.1088/1741-2560/8/2/025005 – ident: 2016071123264944000_iww026v1.15 doi: 10.1109/T-AFFC.2011.7 – ident: 2016071123264944000_iww026v1.17 doi: 10.1016/j.jrp.2005.08.007 – ident: 2016071123264944000_iww026v1.29 doi: 10.1109/TNSRE.2011.2160357 – ident: 2016071123264944000_iww026v1.1 doi: 10.1016/j.math.2011.05.006 – ident: 2016071123264944000_iww026v1.5 doi: 10.1109/TAFFC.2014.2326402 – ident: 2016071123264944000_iww026v1.21 doi: 10.1016/j.biopsycho.2010.03.010 – ident: 2016071123264944000_iww026v1.25 doi: 10.1080/02701367.1989.10607413 – ident: 2016071123264944000_iww026v1.9 – ident: 2016071123264944000_iww026v1.3 – ident: 2016071123264944000_iww026v1.22 doi: 10.1080/10447310902963944 – volume: 12 start-page: 176 year: 2009 ident: 2016071123264944000_iww026v1.37 article-title: Affective e-learning: using ‘emotional’ data to improve learning in pervasive learning environment publication-title: J. Educ. Techno. Soc. – ident: 2016071123264944000_iww026v1.23 doi: 10.1109/TRO.2008.2001362 – ident: 2016071123264944000_iww026v1.34 doi: 10.1111/j.1469-8986.2009.00947.x – ident: 2016071123264944000_iww026v1.35 doi: 10.1016/0301-0511(95)05116-3 – ident: 2016071123264944000_iww026v1.28 doi: 10.1016/j.intcom.2012.04.003 – volume: 29 start-page: 287 year: 2011 ident: 2016071123264944000_iww026v1.11 article-title: Virtual reality based rehabilitation speeds up functional recovery of the upper extremities after stroke: a randomized controlled pilot study in the acute phase of stroke using the Rehabilitation Gaming System publication-title: Restor. Neurol. Neurosci. – ident: 2016071123264944000_iww026v1.19 doi: 10.1016/j.cmpb.2013.02.003 – ident: 2016071123264944000_iww026v1.14 |
| SSID | ssj0007784 |
| Score | 2.2838101 |
| Snippet | Abstract
Classification algorithms are used in affective computing to classify the state of the user and adapt the computer's behaviour, but it is unclear how... |
| SourceID | crossref oup |
| SourceType | Enrichment Source Index Database Publisher |
| StartPage | 271 |
| Title | Absolute and Relative User Perception of Classification Accuracy in an Affective Video Game |
| Volume | 29 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV3da9swEBdZ-rKXrftiXbsi2PYwiltbki370bRby1j2smYU9hB0-gDTkoySrtC_vidLdpItdO1eRBCSQ3S_nO7Od78j5D3LisoKqBLFM54IByYBhv9Hg5OCmbKA0hc4j74VJ2Px5Sw_GwxulqtL5rCvb9bWlfyPVHEO5eqrZB8g2f6hOIGfUb44ooRxvJeMa2ifb2OV4UUg8R7jd_vM9piw0iZceBvZJwUFcddaX136Pu-NT0X2b3WC2tv70Rg72ztWqwwGbdTQF0B0cVsdW0H09vhIH6Lx2UZS7QJux_49fP6hztT5ohymjz7PfofpI88BsRx8wAutz766q6hxJcrocy8Cgcu-DTq2lDyRVeSZjUo4hj2aJV84atTQoSVezizwZv-l9wMnVnOt2_E6ZX_Qa0d_Z93CR2SDoYvBhmSjPhp9_d7f41K2_ar7n9CR21b8APcfhN0r5owvkVyyTk43yZPoVtA6YOQZGdjpc_K0a9lBowZ_QX52kKEIGdpBhnrI0AVk6MzRVcjQDjK0meJW2kOGtpChHjIvyfjzp9PDkyT210g0q-Q8caXKnOQCUtBMqBy0NVJB5tBolhmUtnIqNwVwCVWaG-eJgYRAkxMAmE4Nf0WG09nUviZUFTp1wHUpMi3yzIEr0C8wQijgwjq3RT52xzTRkXze90C5mIQkCD7BI52EI90i7_q1vwLlytpVu3jadyx4868F2-TxAtI7ZDi_vLJv0cKcw26Ewi1p_YJm |
| linkProvider | EBSCOhost |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Absolute+and+Relative+User+Perception+of+Classification+Accuracy+in+an+Affective+Video+Game&rft.jtitle=Interacting+with+computers&rft.au=McCrea%2C+Sean&rft.au=Ger%C5%A1ak%2C+Gregor&rft.au=Novak%2C+Domen&rft.date=2017-03-01&rft.pub=Oxford+University+Press&rft.issn=0953-5438&rft.eissn=1873-7951&rft.volume=29&rft.issue=2&rft.spage=271&rft.epage=286&rft_id=info:doi/10.1093%2Fiwc%2Fiww026&rft.externalDocID=10.1093%2Fiwc%2Fiww026 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0953-5438&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0953-5438&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0953-5438&client=summon |