Nonnaïveté among Amazon Mechanical Turk workers: Consequences and solutions for behavioral researchers
Crowdsourcing services—particularly Amazon Mechanical Turk—have made it easy for behavioral scientists to recruit research participants. However, researchers have overlooked crucial differences between crowdsourcing and traditional recruitment methods that provide unique opportunities and challenges...
Saved in:
| Published in | Behavior research methods Vol. 46; no. 1; pp. 112 - 130 |
|---|---|
| Main Authors | , , |
| Format | Journal Article |
| Language | English |
| Published |
Boston
Springer US
01.03.2014
Springer Nature B.V |
| Subjects | |
| Online Access | Get full text |
Cover
Loading…
| Abstract | Crowdsourcing services—particularly Amazon Mechanical Turk—have made it easy for behavioral scientists to recruit research participants. However, researchers have overlooked crucial differences between crowdsourcing and traditional recruitment methods that provide unique opportunities and challenges. We show that crowdsourced workers are likely to participate across multiple related experiments and that researchers are overzealous in the exclusion of research participants. We describe how both of these problems can be avoided using advanced interface features that also allow prescreening and longitudinal data collection. Using these techniques can minimize the effects of previously ignored drawbacks and expand the scope of crowdsourcing as a tool for psychological research. |
|---|---|
| AbstractList | Crowdsourcing services-particularly Amazon Mechanical Turk-have made it easy for behavioral scientists to recruit research participants. However, researchers have overlooked crucial differences between crowdsourcing and traditional recruitment methods that provide unique opportunities and challenges. We show that crowdsourced workers are likely to participate across multiple related experiments and that researchers are overzealous in the exclusion of research participants. We describe how both of these problems can be avoided using advanced interface features that also allow prescreening and longitudinal data collection. Using these techniques can minimize the effects of previously ignored drawbacks and expand the scope of crowdsourcing as a tool for psychological research. Crowdsourcing services--particularly Amazon Mechanical Turk--have made it easy for behavioral scientists to recruit research participants. However, researchers have overlooked crucial differences between crowdsourcing and traditional recruitment methods that provide unique opportunities and challenges. We show that crowdsourced workers are likely to participate across multiple related experiments and that researchers are overzealous in the exclusion of research participants. We describe how both of these problems can be avoided using advanced interface features that also allow prescreening and longitudinal data collection. Using these techniques can minimize the effects of previously ignored drawbacks and expand the scope of crowdsourcing as a tool for psychological research.Crowdsourcing services--particularly Amazon Mechanical Turk--have made it easy for behavioral scientists to recruit research participants. However, researchers have overlooked crucial differences between crowdsourcing and traditional recruitment methods that provide unique opportunities and challenges. We show that crowdsourced workers are likely to participate across multiple related experiments and that researchers are overzealous in the exclusion of research participants. We describe how both of these problems can be avoided using advanced interface features that also allow prescreening and longitudinal data collection. Using these techniques can minimize the effects of previously ignored drawbacks and expand the scope of crowdsourcing as a tool for psychological research. |
| Author | Paolacci, Gabriele Chandler, Jesse Mueller, Pam |
| Author_xml | – sequence: 1 givenname: Jesse surname: Chandler fullname: Chandler, Jesse email: jjchandl@umich.edu organization: Woodrow Wilson School of Public Affairs, Princeton University – sequence: 2 givenname: Pam surname: Mueller fullname: Mueller, Pam organization: Department of Psychology, Princeton University – sequence: 3 givenname: Gabriele surname: Paolacci fullname: Paolacci, Gabriele organization: Rotterdam School of Management, Erasmus University |
| BackLink | https://www.ncbi.nlm.nih.gov/pubmed/23835650$$D View this record in MEDLINE/PubMed |
| BookMark | eNqNks1OFjEUhhuDkR-9ADemiRs3o_2ddtyRL6ImqBtcN53OKd_ATIvtDATuiJUXwY3ZyQeEkEhYtWme5_S0591FWyEGQOgtJR-5kvpTplwwXRHKK8JrWakXaIdKKSoumd56sN9GuzmfEMI1o-IV2mZcc1lLsoPWP2MI9ubvOUw319iOMRzj_dFexYB_gFvb0Ds74KM5neKLmE4h5c94FUOGPzMEBxnb0OEch3nqyyn2MeEW1va8j6l4CTLY5NZFe41eejtkeHO77qHfB1-OVt-qw19fv6_2DysnVT1V0hIOraKgveOs5Z5a4r2yHZHaK0mY9y24ulPAtGiEI20rOi2ZpF6LlnC-hz5s6p6lWHrMkxn77GAYbIA4Z0MlF5o2pdIzUFa-lapGPQMlXArN9IK-f4SexDmF8mbD6lrWVBPRPEVRKVQjJJW0UO9uqbkdoTNnqR9tujR38ysA3QAuxZwT-HuEErNkxGwyYkpGzJIRszSoHjmun-wyvynZfnjSZBszl1vCMaQHTf9X-gdkXs_x |
| CitedBy_id | crossref_primary_10_1007_s11002_024_09718_6 crossref_primary_10_2200_S00904ED1V01Y201903ICR066 crossref_primary_10_1146_annurev_clinpsy_021815_093623 crossref_primary_10_1017_S0007123418000327 crossref_primary_10_2139_ssrn_2182588 crossref_primary_10_1080_10852352_2019_1603676 crossref_primary_10_1177_13591053221079177 crossref_primary_10_1108_EJM_07_2017_0491 crossref_primary_10_1177_1556264620927583 crossref_primary_10_3758_s13428_017_0882_x crossref_primary_10_1007_s10826_019_01379_8 crossref_primary_10_1093_jopart_muaa049 crossref_primary_10_3897_rio_2_e8848 crossref_primary_10_1080_00223891_2017_1339711 crossref_primary_10_3758_s13428_013_0434_y crossref_primary_10_1177_0146167215612743 crossref_primary_10_1177_01461672221103058 crossref_primary_10_1016_j_tics_2017_06_007 crossref_primary_10_3758_s13428_021_01588_4 crossref_primary_10_1177_2378023117737193 crossref_primary_10_1002_job_2118 crossref_primary_10_1080_07481756_2022_2073895 crossref_primary_10_1162_rest_a_01240 crossref_primary_10_1038_s41562_018_0451_z crossref_primary_10_1177_2515245917740427 crossref_primary_10_1080_1068316X_2018_1438432 crossref_primary_10_1111_pops_12386 crossref_primary_10_2308_accr_51447 crossref_primary_10_1177_0022427819886111 crossref_primary_10_1177_0003122417747290 crossref_primary_10_1016_j_paid_2016_03_091 crossref_primary_10_1177_0190272518765134 crossref_primary_10_1016_j_jesp_2015_09_010 crossref_primary_10_1177_1354068817722671 crossref_primary_10_2196_diabetes_7473 crossref_primary_10_1108_JRIM_11_2019_0179 crossref_primary_10_1371_journal_pone_0246689 crossref_primary_10_1080_13546783_2022_2146191 crossref_primary_10_2139_ssrn_2734405 crossref_primary_10_3389_fpsyg_2014_01027 crossref_primary_10_1080_01900692_2019_1659816 crossref_primary_10_1371_journal_pone_0157732 crossref_primary_10_1177_1073191119860903 crossref_primary_10_1002_nvsm_1619 crossref_primary_10_1016_j_jesp_2022_104377 crossref_primary_10_1007_s12144_015_9403_1 crossref_primary_10_1123_ijsc_2021_0068 crossref_primary_10_1016_j_paid_2015_08_051 crossref_primary_10_1080_02664763_2018_1547692 crossref_primary_10_1007_s10869_016_9458_5 crossref_primary_10_1080_10615806_2018_1521958 crossref_primary_10_1007_s40750_022_00196_6 crossref_primary_10_1016_j_trf_2018_04_001 crossref_primary_10_1177_1073191116662909 crossref_primary_10_1093_fpa_orw012 crossref_primary_10_1177_0146167220946195 crossref_primary_10_1016_j_ijresmar_2021_05_001 crossref_primary_10_1016_j_ijinfomgt_2021_102361 crossref_primary_10_1080_13645579_2018_1563966 crossref_primary_10_1177_0001839218793128 crossref_primary_10_16995_labphon_8778 crossref_primary_10_1080_02650487_2020_1827895 crossref_primary_10_1080_03637751_2016_1246348 crossref_primary_10_1016_j_socscimed_2021_114562 crossref_primary_10_1080_13674676_2018_1486394 crossref_primary_10_12688_f1000research_20183_3 crossref_primary_10_1080_10508422_2020_1860767 crossref_primary_10_12688_f1000research_20183_4 crossref_primary_10_3758_s13428_018_1028_5 crossref_primary_10_12688_f1000research_20183_1 crossref_primary_10_1016_j_pubrev_2018_02_001 crossref_primary_10_12688_f1000research_20183_2 crossref_primary_10_3156_jsoft_35_4_769 crossref_primary_10_1177_0734016820978827 crossref_primary_10_1080_00913367_2016_1277380 crossref_primary_10_1016_j_sleh_2021_10_004 crossref_primary_10_1177_1079063220951191 crossref_primary_10_3758_s13428_020_01535_9 crossref_primary_10_1080_00031305_2015_1077728 crossref_primary_10_1371_journal_pone_0154467 crossref_primary_10_5964_jspp_v3i2_434 crossref_primary_10_1086_690450 crossref_primary_10_1371_journal_pone_0156527 crossref_primary_10_1080_10967494_2016_1276493 crossref_primary_10_1177_0963721414531598 crossref_primary_10_2139_ssrn_2609920 crossref_primary_10_1017_S0003055418000837 crossref_primary_10_1080_1554477X_2021_1941630 crossref_primary_10_1044_2019_AJSLP_18_0196 crossref_primary_10_1111_josh_13134 crossref_primary_10_1177_0192513X211029264 crossref_primary_10_1177_0956797617754219 crossref_primary_10_1007_s41055_019_00051_7 crossref_primary_10_1080_00224545_2023_2169101 crossref_primary_10_1177_0146167216684132 crossref_primary_10_1177_14777509231211666 crossref_primary_10_1111_poms_12841 crossref_primary_10_1007_s11116_019_10031_9 crossref_primary_10_1017_iop_2015_130 crossref_primary_10_1027_1614_0001_a000360 crossref_primary_10_1016_j_jcrimjus_2023_102074 crossref_primary_10_1016_j_rmal_2022_100031 crossref_primary_10_1891_RE_22_20 crossref_primary_10_1177_0146167216662868 crossref_primary_10_1016_j_apergo_2017_05_001 crossref_primary_10_5964_miss_13619 crossref_primary_10_1016_j_obhdp_2020_03_001 crossref_primary_10_1111_joca_12344 crossref_primary_10_1007_s11846_022_00602_z crossref_primary_10_1080_10447318_2022_2144833 crossref_primary_10_1177_09732586241252201 crossref_primary_10_3389_frbhe_2023_1164259 crossref_primary_10_1080_13546783_2018_1507949 crossref_primary_10_1111_ctr_12449 crossref_primary_10_1007_s13164_017_0374_z crossref_primary_10_1177_19375867221104248 crossref_primary_10_1177_0146167220921713 crossref_primary_10_1177_1948550617703168 crossref_primary_10_1016_j_metip_2020_100022 crossref_primary_10_1177_1073191117719513 crossref_primary_10_1017_S0007123416000326 crossref_primary_10_1108_JCOM_12_2018_0132 crossref_primary_10_1016_j_joep_2020_102268 crossref_primary_10_1016_j_jrp_2019_103856 crossref_primary_10_1007_s40593_019_00188_w crossref_primary_10_1108_IJLM_03_2023_0109 crossref_primary_10_1111_emip_12485 crossref_primary_10_1002_acp_3764 crossref_primary_10_1017_S1930297500003405 crossref_primary_10_1111_ijsa_12302 crossref_primary_10_1002_jts_22303 crossref_primary_10_1016_j_jesp_2015_11_004 crossref_primary_10_1027_1618_3169_a000636 crossref_primary_10_1108_EJIM_12_2021_0597 crossref_primary_10_1080_07317115_2016_1198856 crossref_primary_10_1016_j_aei_2025_103236 crossref_primary_10_3758_s13428_017_0881_y crossref_primary_10_1371_journal_pone_0239707 crossref_primary_10_1177_2515245919838479 crossref_primary_10_1093_jopart_muv021 crossref_primary_10_1145_3326337 crossref_primary_10_1177_2053168015622072 crossref_primary_10_1371_journal_pone_0140383 crossref_primary_10_1027_1864_1105_a000404 crossref_primary_10_1007_s10964_016_0490_6 crossref_primary_10_1177_2050157920958437 crossref_primary_10_1007_s12119_018_9543_1 crossref_primary_10_1177_2053168019901185 crossref_primary_10_1111_jori_12402 crossref_primary_10_1080_08824096_2020_1867090 crossref_primary_10_12688_f1000research_25573_1 crossref_primary_10_1016_j_ijintrel_2018_03_007 crossref_primary_10_1038_s41598_023_46048_5 crossref_primary_10_12688_f1000research_25573_2 crossref_primary_10_1111_jopy_12237 crossref_primary_10_1016_j_trf_2019_05_008 crossref_primary_10_1080_07481756_2021_1955215 crossref_primary_10_1177_2053168015604648 crossref_primary_10_2139_ssrn_3224670 crossref_primary_10_1177_0956797615585115 crossref_primary_10_1016_j_drugalcdep_2018_10_007 crossref_primary_10_1075_jlp_20032_har crossref_primary_10_2139_ssrn_3978470 crossref_primary_10_2139_ssrn_2733983 crossref_primary_10_1080_07481756_2024_2344000 crossref_primary_10_1016_j_cub_2022_02_039 crossref_primary_10_1177_0956797616650875 crossref_primary_10_1007_s10584_021_02975_8 crossref_primary_10_1080_17457289_2016_1270287 crossref_primary_10_1111_fare_12504 crossref_primary_10_1111_ssqu_12477 crossref_primary_10_1177_0146167214541659 crossref_primary_10_1016_j_chb_2017_08_038 crossref_primary_10_1016_j_chb_2020_106266 crossref_primary_10_2139_ssrn_2477454 crossref_primary_10_1177_01461672241276562 crossref_primary_10_1177_0146167216649609 crossref_primary_10_1215_15476715_11199970 crossref_primary_10_1016_j_jesp_2018_05_006 crossref_primary_10_1177_0093854816668916 crossref_primary_10_2308_HORIZONS_18_052 crossref_primary_10_1177_20413866221097571 crossref_primary_10_1111_jopy_12256 crossref_primary_10_1002_ejsp_2526 crossref_primary_10_1016_j_jcbs_2019_02_002 crossref_primary_10_1080_10447318_2019_1709339 crossref_primary_10_1371_journal_pone_0211890 crossref_primary_10_1080_02642069_2021_2006644 crossref_primary_10_1111_deci_12403 crossref_primary_10_1177_00302228221090754 crossref_primary_10_1016_j_joep_2018_12_011 crossref_primary_10_1038_s41467_019_09075_3 crossref_primary_10_1177_13684302211001946 crossref_primary_10_1002_nvsm_1676 crossref_primary_10_20982_tqmp_16_5_p472 crossref_primary_10_3389_fpsyg_2017_00295 crossref_primary_10_2139_ssrn_4147490 crossref_primary_10_2139_ssrn_4147493 crossref_primary_10_1111_famp_12329 crossref_primary_10_3390_su15108097 crossref_primary_10_1111_bjso_12417 crossref_primary_10_5964_jspp_8379 crossref_primary_10_1016_j_biopsych_2017_12_017 crossref_primary_10_1111_ijsa_12220 crossref_primary_10_1108_INTR_05_2021_0269 crossref_primary_10_1111_ssqu_12338 crossref_primary_10_3389_fpos_2022_817309 crossref_primary_10_1111_ssqu_12336 crossref_primary_10_1177_2158244019876268 crossref_primary_10_3758_s13428_023_02321_z crossref_primary_10_1007_s10508_021_01911_2 crossref_primary_10_1177_0149206320969787 crossref_primary_10_1016_j_cedpsych_2016_08_004 crossref_primary_10_7717_peerj_8339 crossref_primary_10_1515_jdpa_2019_0010 crossref_primary_10_1007_s10936_024_10078_5 crossref_primary_10_1371_journal_pone_0221676 crossref_primary_10_1007_s40881_015_0007_1 crossref_primary_10_1027_1618_3169_a000562 crossref_primary_10_1007_s10461_015_1107_x crossref_primary_10_3390_ijerph16050881 crossref_primary_10_1007_s11948_017_9886_2 crossref_primary_10_3389_fpsyg_2019_00004 crossref_primary_10_5964_jspp_v3i1_117 crossref_primary_10_1057_s41284_021_00296_0 crossref_primary_10_1016_j_janxdis_2022_102602 crossref_primary_10_5334_joc_85 crossref_primary_10_1177_15553434241296170 crossref_primary_10_1016_j_bpsc_2023_02_004 crossref_primary_10_1016_j_jom_2017_06_001 crossref_primary_10_1016_j_jesp_2018_02_003 crossref_primary_10_1371_journal_pone_0153419 crossref_primary_10_1017_S000712341700014X crossref_primary_10_1177_0146167218771000 crossref_primary_10_1177_2329488420979657 crossref_primary_10_1080_15298868_2016_1194314 crossref_primary_10_2139_ssrn_4991815 crossref_primary_10_1002_cb_1886 crossref_primary_10_1080_24732850_2018_1439142 crossref_primary_10_1177_02762366241247779 crossref_primary_10_1016_j_chb_2016_11_005 crossref_primary_10_1093_jcr_ucx053 crossref_primary_10_1177_0146167216688211 crossref_primary_10_15195_v1_a19 crossref_primary_10_1016_j_cub_2018_10_053 crossref_primary_10_1080_10447318_2022_2150691 crossref_primary_10_1108_EJM_07_2018_0468 crossref_primary_10_1016_j_chb_2020_106585 crossref_primary_10_1080_00223891_2020_1729773 crossref_primary_10_1016_j_technovation_2023_102754 crossref_primary_10_1017_S1930297500008007 crossref_primary_10_1016_j_appdev_2016_02_005 crossref_primary_10_1016_j_jesp_2015_08_005 crossref_primary_10_1016_j_actpsy_2019_102867 crossref_primary_10_1177_0049124119826151 crossref_primary_10_1111_asap_12081 crossref_primary_10_1177_16094069231162674 crossref_primary_10_3389_fpsyg_2019_02480 crossref_primary_10_1007_s10826_015_0318_0 crossref_primary_10_1093_jopart_muaa010 crossref_primary_10_1093_jcr_ucx047 crossref_primary_10_1371_journal_pone_0235883 crossref_primary_10_2139_ssrn_3767695 crossref_primary_10_1080_02650487_2017_1348329 crossref_primary_10_1073_pnas_2009641117 crossref_primary_10_1177_0146167215607195 crossref_primary_10_1007_s10606_018_9313_5 crossref_primary_10_1016_j_ijresmar_2018_01_003 crossref_primary_10_1016_j_jesp_2017_01_006 crossref_primary_10_1177_00220574221101378 crossref_primary_10_1108_MD_05_2017_0537 crossref_primary_10_1016_j_jbef_2021_100512 crossref_primary_10_1017_XPS_2019_35 crossref_primary_10_2139_ssrn_2884409 crossref_primary_10_3758_s13428_021_01694_3 crossref_primary_10_1108_EJM_07_2018_0470 crossref_primary_10_1177_0149206318757019 crossref_primary_10_1007_s12671_015_0485_5 crossref_primary_10_1080_01419870_2016_1260751 crossref_primary_10_1002_mar_21531 crossref_primary_10_1037_gpr0000027 crossref_primary_10_1093_jcr_ucy041 crossref_primary_10_1002_nml_21577 crossref_primary_10_1177_08997640241254079 crossref_primary_10_3389_fpsyg_2017_01062 crossref_primary_10_1145_2928269 crossref_primary_10_2308_bria_52103 crossref_primary_10_1080_08927936_2023_2204640 crossref_primary_10_1111_bjso_12462 crossref_primary_10_1007_s12369_023_01041_w crossref_primary_10_1080_23311908_2019_1674091 crossref_primary_10_1146_annurev_orgpsych_120920_052946 crossref_primary_10_2139_ssrn_2324255 crossref_primary_10_5964_jspp_v2i1_280 crossref_primary_10_1080_10410236_2019_1700882 crossref_primary_10_1080_00913367_2020_1806155 crossref_primary_10_1111_apps_12302 crossref_primary_10_1177_1745691617706516 crossref_primary_10_1007_s10608_014_9597_9 crossref_primary_10_1093_workar_waaa005 crossref_primary_10_1111_ssqu_12521 crossref_primary_10_3758_s13428_022_01955_9 crossref_primary_10_1371_journal_pone_0278677 crossref_primary_10_1177_17456916241234328 crossref_primary_10_5964_jspp_v1i1_184 crossref_primary_10_1111_psq_12298 crossref_primary_10_3389_fpsyg_2018_01399 crossref_primary_10_1177_00438200221107018 crossref_primary_10_1016_j_jbusvent_2018_10_008 crossref_primary_10_2139_ssrn_2405202 crossref_primary_10_1080_10502556_2020_1768491 crossref_primary_10_1080_10871209_2019_1605636 crossref_primary_10_1111_1745_9125_12222 crossref_primary_10_1371_journal_pone_0112042 crossref_primary_10_5964_jspp_v6i1_810 crossref_primary_10_1108_CCIJ_12_2023_0196 crossref_primary_10_1016_j_concog_2021_103215 crossref_primary_10_3389_fpsyg_2019_02833 crossref_primary_10_1007_s12119_018_9497_3 crossref_primary_10_1007_s11116_018_9962_8 crossref_primary_10_1007_s10551_016_3095_9 crossref_primary_10_1017_iop_2015_23 crossref_primary_10_1108_IMDS_09_2023_0608 crossref_primary_10_3758_s13428_022_02005_0 crossref_primary_10_1017_iop_2015_21 crossref_primary_10_1177_0003122419879101 crossref_primary_10_2139_ssrn_3082952 crossref_primary_10_1017_iop_2015_20 crossref_primary_10_1016_j_jbusres_2021_11_065 crossref_primary_10_1016_j_psychsport_2024_102719 crossref_primary_10_1080_17524032_2019_1575258 crossref_primary_10_4992_jjpsy_95_22336 crossref_primary_10_1093_jopart_muz044 crossref_primary_10_1371_journal_pone_0244958 crossref_primary_10_1177_00811750211014242 crossref_primary_10_1371_journal_pone_0118715 crossref_primary_10_52399_001c_22019 crossref_primary_10_1111_pcn_13279 crossref_primary_10_1177_1948550620972296 crossref_primary_10_1016_j_jbusres_2015_12_001 crossref_primary_10_1177_1087054715597471 crossref_primary_10_1038_s41562_020_0849_2 crossref_primary_10_1016_j_jmp_2017_08_005 crossref_primary_10_1080_14719037_2018_1529878 crossref_primary_10_1111_apps_12124 crossref_primary_10_1080_00913367_2016_1230838 crossref_primary_10_1371_journal_pone_0285267 crossref_primary_10_1108_JCOM_06_2020_0057 crossref_primary_10_1080_15377857_2017_1411860 crossref_primary_10_1177_0003122420969399 crossref_primary_10_1016_j_eatbeh_2023_101824 crossref_primary_10_1016_j_jbtep_2016_08_018 crossref_primary_10_20867_thm_30_4_6 crossref_primary_10_1017_S1930297500007622 crossref_primary_10_1007_s12144_017_9733_2 crossref_primary_10_1080_00913367_2016_1225233 crossref_primary_10_3758_s13414_017_1361_2 crossref_primary_10_1080_17430437_2020_1804116 crossref_primary_10_1007_s10826_016_0598_z crossref_primary_10_1007_s11747_014_0389_2 crossref_primary_10_1007_s10683_017_9527_2 crossref_primary_10_1111_fare_12206 crossref_primary_10_2308_isys_51980 crossref_primary_10_1177_2053168017751250 crossref_primary_10_1002_ab_21905 crossref_primary_10_1111_1911_3838_12277 crossref_primary_10_2139_ssrn_4163676 crossref_primary_10_1111_1745_9125_12319 crossref_primary_10_1371_journal_pone_0205462 crossref_primary_10_1093_poq_nfw030 crossref_primary_10_1080_15332667_2023_2257099 crossref_primary_10_7717_peerj_10318 crossref_primary_10_2139_ssrn_3765448 crossref_primary_10_1145_3412374 crossref_primary_10_1155_2022_6432445 crossref_primary_10_2139_ssrn_3272468 crossref_primary_10_1007_s10551_018_3811_8 crossref_primary_10_1080_16184742_2018_1446994 crossref_primary_10_1177_1948550614549482 crossref_primary_10_1027_1864_9335_a000292 crossref_primary_10_1080_0144929X_2020_1869307 crossref_primary_10_1177_1368430216684645 crossref_primary_10_1080_15332845_2022_2031606 crossref_primary_10_2139_ssrn_2797672 crossref_primary_10_1002_hbe2_296 crossref_primary_10_5406_amerjpsyc_134_2_0143 crossref_primary_10_1080_1060586X_2020_1730121 crossref_primary_10_1016_j_wneu_2021_10_185 crossref_primary_10_2139_ssrn_5157774 crossref_primary_10_1098_rsos_200793 crossref_primary_10_1111_ajps_12396 crossref_primary_10_1007_s11109_017_9417_5 crossref_primary_10_1016_j_paid_2020_110312 crossref_primary_10_1016_j_josat_2023_209011 crossref_primary_10_1080_14626268_2017_1322988 crossref_primary_10_1016_j_jbef_2019_03_005 crossref_primary_10_1080_09687637_2020_1854688 crossref_primary_10_1080_15332667_2024_2368323 crossref_primary_10_1111_apps_12108 crossref_primary_10_1017_S104909651500116X crossref_primary_10_1111_spc3_12368 crossref_primary_10_1017_S0003055416000058 crossref_primary_10_1016_j_ssresearch_2018_02_004 crossref_primary_10_1111_ijcs_12878 crossref_primary_10_1371_journal_pone_0130834 crossref_primary_10_1016_j_jbusres_2014_09_036 crossref_primary_10_2139_ssrn_2842577 crossref_primary_10_1080_00208825_2021_1927311 crossref_primary_10_2139_ssrn_2924876 crossref_primary_10_1016_j_cognition_2020_104342 crossref_primary_10_1177_0894439316650163 crossref_primary_10_1080_0144929X_2025_2471908 crossref_primary_10_3390_su12239942 crossref_primary_10_1016_j_paid_2019_109800 crossref_primary_10_1111_ssqu_12954 crossref_primary_10_2139_ssrn_3339102 crossref_primary_10_5964_jspp_v8i1_1187 crossref_primary_10_1177_19375867221143104 crossref_primary_10_1080_20445911_2018_1564756 crossref_primary_10_3758_s13428_015_0642_8 crossref_primary_10_1080_15256480_2022_2103485 crossref_primary_10_1177_0192513X18821395 crossref_primary_10_3389_fpsyg_2017_01359 crossref_primary_10_1027_1015_5759_a000575 crossref_primary_10_3758_s13423_017_1348_y crossref_primary_10_1016_j_jcomdis_2016_07_001 crossref_primary_10_1016_j_promfg_2015_07_514 crossref_primary_10_3169_itej_75_480 crossref_primary_10_1038_s41598_022_22995_3 crossref_primary_10_1080_10463283_2018_1542902 crossref_primary_10_3758_s13428_025_02618_1 crossref_primary_10_1002_ejsp_2326 crossref_primary_10_3389_fpsyg_2019_02646 crossref_primary_10_1017_S0140525X22002874 crossref_primary_10_5964_jspp_v4i2_245 crossref_primary_10_1080_17457289_2019_1651321 crossref_primary_10_1177_2059799116672879 crossref_primary_10_3758_s13428_023_02220_3 crossref_primary_10_1016_j_jretai_2019_11_001 crossref_primary_10_1016_j_ssresearch_2016_04_014 crossref_primary_10_2139_ssrn_3859779 crossref_primary_10_3389_fpsyg_2017_00378 crossref_primary_10_1177_10648046221075942 crossref_primary_10_1007_s10826_017_0741_5 crossref_primary_10_5465_amd_2017_0002 crossref_primary_10_1371_journal_pone_0190954 crossref_primary_10_2196_resprot_8071 crossref_primary_10_1016_j_socec_2023_102106 crossref_primary_10_1108_ARJ_05_2019_0090 crossref_primary_10_1177_19485506211060115 crossref_primary_10_2139_ssrn_4143504 crossref_primary_10_1016_j_ijlp_2019_101536 crossref_primary_10_1017_XPS_2017_25 crossref_primary_10_1146_annurev_psych_010814_015321 crossref_primary_10_1080_15564886_2023_2183917 crossref_primary_10_1007_s10826_015_0244_1 crossref_primary_10_1177_0146167219841638 crossref_primary_10_1111_famp_12657 crossref_primary_10_3758_s13428_015_0578_z crossref_primary_10_1111_famp_12516 crossref_primary_10_1007_s12144_024_06669_z crossref_primary_10_1007_s10551_019_04370_x crossref_primary_10_1016_j_jenvp_2020_101524 crossref_primary_10_1080_15252019_2020_1841048 crossref_primary_10_1177_0003122419832497 crossref_primary_10_1017_S1930297500007683 crossref_primary_10_1371_journal_pone_0290818 crossref_primary_10_1016_j_jsis_2016_06_001 crossref_primary_10_1111_1745_9125_12129 crossref_primary_10_1093_oxrep_gru035 crossref_primary_10_7717_peerj_11559 crossref_primary_10_1080_1068316X_2017_1409895 crossref_primary_10_7717_peerj_1058 crossref_primary_10_1177_0149206318811569 crossref_primary_10_1007_s10940_019_09436_7 crossref_primary_10_1016_j_chb_2018_02_005 crossref_primary_10_1017_S1930297500003429 crossref_primary_10_1111_cogs_13041 crossref_primary_10_1080_15309576_2018_1474770 crossref_primary_10_1016_j_jbef_2017_12_004 crossref_primary_10_1371_journal_pone_0213142 crossref_primary_10_2139_ssrn_3727227 crossref_primary_10_1016_j_ijresmar_2017_10_001 crossref_primary_10_1108_EDI_04_2019_0140 crossref_primary_10_1111_pops_12411 crossref_primary_10_20982_tqmp_13_3_p174 crossref_primary_10_1111_add_15032 crossref_primary_10_1093_poq_nfad037 crossref_primary_10_1002_ajim_23619 crossref_primary_10_1108_JMP_07_2020_0413 crossref_primary_10_2139_ssrn_2956147 crossref_primary_10_1016_j_addbeh_2015_10_007 crossref_primary_10_1016_j_actpsy_2019_01_008 crossref_primary_10_1016_j_chb_2015_08_031 crossref_primary_10_1016_j_jbef_2023_100790 crossref_primary_10_1177_08862605241301795 crossref_primary_10_1111_camh_12665 crossref_primary_10_1080_10494820_2024_2408554 crossref_primary_10_1525_collabra_185 crossref_primary_10_2308_bria_51977 crossref_primary_10_1016_j_concog_2023_103502 crossref_primary_10_1177_2153368717749879 crossref_primary_10_2139_ssrn_2972406 crossref_primary_10_1016_j_jbef_2019_04_010 crossref_primary_10_1177_2329488420929748 crossref_primary_10_1371_journal_pone_0140806 crossref_primary_10_1080_08923647_2017_1258660 crossref_primary_10_1525_collabra_181 crossref_primary_10_1111_bjep_12162 crossref_primary_10_1080_13645579_2021_1961187 crossref_primary_10_1007_s40881_021_00114_8 crossref_primary_10_1371_journal_pone_0149989 crossref_primary_10_1007_s11145_018_9872_3 crossref_primary_10_3389_fcomm_2019_00007 crossref_primary_10_1080_16184742_2024_2329157 crossref_primary_10_5334_irsp_591 crossref_primary_10_1007_s12599_022_00754_y crossref_primary_10_1080_20445911_2020_1766472 crossref_primary_10_1080_15332861_2020_1740962 crossref_primary_10_1177_1359105319845130 crossref_primary_10_1111_aphw_12536 crossref_primary_10_1016_j_evolhumbehav_2018_05_010 crossref_primary_10_1016_j_paid_2019_109702 crossref_primary_10_1080_15298868_2016_1269669 crossref_primary_10_1002_pits_22275 crossref_primary_10_1080_08838151_2015_1093481 crossref_primary_10_15195_v4_a19 crossref_primary_10_1111_jcom_12260 crossref_primary_10_1080_10810730_2015_1058439 crossref_primary_10_1016_j_jenvp_2019_101351 crossref_primary_10_1108_JCM_03_2022_5273 crossref_primary_10_17744_mehc_43_2_05 crossref_primary_10_1016_j_metip_2020_100032 crossref_primary_10_1162_edfp_a_00271 crossref_primary_10_1177_00811750241312226 crossref_primary_10_3758_s13428_024_02533_x crossref_primary_10_1177_1948550617698203 crossref_primary_10_1007_s12369_023_01046_5 crossref_primary_10_1111_lasr_12264 crossref_primary_10_1007_s11292_023_09570_0 crossref_primary_10_1017_S0140525X18000808 crossref_primary_10_1016_j_paid_2019_02_015 crossref_primary_10_1080_13607863_2015_1072800 crossref_primary_10_1016_j_chb_2017_02_054 crossref_primary_10_1016_j_jebo_2018_06_011 crossref_primary_10_1016_j_obhdp_2015_08_003 crossref_primary_10_1016_j_paid_2014_11_017 crossref_primary_10_1016_j_jesp_2018_09_003 crossref_primary_10_5964_jspp_v7i1_955 crossref_primary_10_1017_dsj_2024_23 crossref_primary_10_1177_01461672211005883 crossref_primary_10_3758_s13428_021_01665_8 crossref_primary_10_1177_10738584211017018 crossref_primary_10_1080_02699931_2024_2421400 crossref_primary_10_1007_s00426_023_01832_w crossref_primary_10_3390_brainsci10040251 crossref_primary_10_2139_ssrn_2807497 crossref_primary_10_1016_j_bushor_2016_02_004 crossref_primary_10_1007_s11211_017_0287_0 crossref_primary_10_1177_2515245918810225 crossref_primary_10_3102_0013189X17725519 crossref_primary_10_1111_1467_8551_12168 crossref_primary_10_1007_s40881_021_00112_w crossref_primary_10_1016_j_ijresmar_2020_08_002 crossref_primary_10_1016_j_ijresmar_2020_08_001 crossref_primary_10_2139_ssrn_2975829 crossref_primary_10_1080_1062726X_2020_1861949 crossref_primary_10_1016_j_socec_2024_102241 crossref_primary_10_2139_ssrn_2835408 crossref_primary_10_1371_journal_pone_0100662 crossref_primary_10_1177_2325967117695788 crossref_primary_10_1111_emre_12701 crossref_primary_10_1002_bdm_2377 crossref_primary_10_1016_j_paid_2020_109812 crossref_primary_10_1016_j_joep_2022_102496 crossref_primary_10_3758_s13428_018_1103_y crossref_primary_10_1109_TVCG_2017_2744298 crossref_primary_10_1016_j_joep_2024_102708 crossref_primary_10_2334_josnusd_18_0068 crossref_primary_10_1007_s12144_020_00990_z crossref_primary_10_1080_02699931_2018_1504747 crossref_primary_10_1111_1467_8551_12396 crossref_primary_10_1016_j_paid_2019_109632 crossref_primary_10_1038_s41598_022_12163_y crossref_primary_10_1016_j_jenvp_2023_102092 crossref_primary_10_1016_j_jesp_2019_103819 crossref_primary_10_1016_j_jmpt_2022_02_004 crossref_primary_10_1080_13218719_2020_1751741 crossref_primary_10_1177_0047287519875820 crossref_primary_10_1007_s12671_024_02315_8 crossref_primary_10_1108_PIJPSM_12_2018_0192 crossref_primary_10_1016_j_actpsy_2023_103944 crossref_primary_10_1111_cogs_13476 crossref_primary_10_3758_s13428_017_0984_5 crossref_primary_10_1016_j_jrp_2021_104131 crossref_primary_10_1016_j_jad_2020_03_129 crossref_primary_10_1007_s10869_017_9514_9 crossref_primary_10_5465_amj_2015_0018 crossref_primary_10_1007_s10551_023_05604_9 crossref_primary_10_1073_pnas_1501592112 crossref_primary_10_1080_15298868_2015_1037791 crossref_primary_10_1111_jasp_12580 crossref_primary_10_17352_jtsd_000001 crossref_primary_10_1177_0190272519863424 crossref_primary_10_1177_2158244017712774 crossref_primary_10_1080_00913367_2022_2079026 crossref_primary_10_1016_j_heliyon_2024_e40624 crossref_primary_10_2478_ijm_2020_0006 crossref_primary_10_1371_journal_pone_0252604 crossref_primary_10_1371_journal_pone_0250303 crossref_primary_10_1002_jcad_12369 crossref_primary_10_1086_707010 crossref_primary_10_1016_j_jesp_2017_06_012 crossref_primary_10_1080_02673037_2021_1935767 crossref_primary_10_1111_ajpy_12110 crossref_primary_10_1038_s41398_024_03065_w crossref_primary_10_3758_s13428_014_0471_1 crossref_primary_10_1007_s11747_024_01046_5 crossref_primary_10_1007_s10826_015_0126_6 crossref_primary_10_1016_j_jpsychires_2019_04_023 crossref_primary_10_5334_pb_1064 crossref_primary_10_1177_2158244015584617 crossref_primary_10_1016_j_ypmed_2017_07_006 crossref_primary_10_2139_ssrn_2599513 crossref_primary_10_1016_j_jcrimjus_2025_102375 crossref_primary_10_1177_1557085118773398 crossref_primary_10_2139_ssrn_2594183 crossref_primary_10_1007_s10862_023_10080_6 crossref_primary_10_1016_j_jebo_2021_03_029 crossref_primary_10_1016_j_jbusres_2021_11_054 crossref_primary_10_1111_jels_12211 crossref_primary_10_2139_ssrn_2654056 crossref_primary_10_5334_joc_259 crossref_primary_10_1027_1864_9335_a000256 crossref_primary_10_1016_j_chb_2017_03_048 crossref_primary_10_1016_j_psychres_2016_03_014 crossref_primary_10_1108_IJEBR_07_2022_0629 crossref_primary_10_1177_0091647119837013 crossref_primary_10_2139_ssrn_2807176 crossref_primary_10_1016_j_jesp_2017_06_005 crossref_primary_10_1017_XPS_2015_19 |
| Cites_doi | 10.1177/0956797611417632 10.1207/s15327752jpa4803_13 10.1016/j.jesp.2009.03.009 10.1037/0003-066X.59.2.93 10.2139/ssrn.2100601 10.1007/978-1-4614-8806-4_30 10.1257/089533005775196732 10.1016/B978-012099980-4/50006-X 10.1017/S1930297500002205 10.1111/j.1467-9280.1990.tb00226.x 10.1145/1837885.1837889 10.1126/science.321.5895.1443a 10.1111/j.1365-2966.2008.13689.x 10.1037/h0030073 10.3758/s13428-012-0250-9 10.1002/bdm.1753 10.3758/s13428-011-0124-6 10.1016/j.jesp.2013.01.010 10.1177/2167702612469015 10.1037/a0025391 10.1126/science.1160379 10.1076/clin.13.3.283.1743 10.1086/208604 10.1016/j.jrp.2004.09.009 10.1145/1753326.1753543 10.1037/a0019106 10.1177/0956797610391098 10.1016/0149-7189(95)00037-2 10.1016/0010-0285(75)90021-3 10.1037/a0028857 10.1016/B978-012099980-4/50003-4 10.1177/0146167208331255 10.1177/1745691610393980 10.1111/j.1559-1816.2008.00362.x 10.1037/h0028852 10.1111/j.1551-6709.2011.01210.x 10.1007/s10683-011-9273-9 10.1016/j.jtbi.2011.03.004 10.1037/0022-3514.69.5.797 10.3758/BRM.41.4.1149 10.1038/nature09304 10.1093/pan/mpr057 10.3758/s13428-010-0039-7 10.1037/h0025907 10.1145/1753326.1753688 10.1016/0022-1031(73)90009-7 10.1016/0022-1031(66)90087-4 10.2139/ssrn.2100631 10.3758/s13428-011-0081-0 10.1016/B978-012099980-4/50005-8 10.1177/0193841X9001400608 10.1145/1357054.1357127 10.1371/journal.pone.0016836 |
| ContentType | Journal Article |
| Copyright | Psychonomic Society, Inc. 2013 Copyright Springer Science & Business Media Mar 2014 Psychonomic Society, Inc. 2013. |
| Copyright_xml | – notice: Psychonomic Society, Inc. 2013 – notice: Copyright Springer Science & Business Media Mar 2014 – notice: Psychonomic Society, Inc. 2013. |
| DBID | AAYXX CITATION CGR CUY CVF ECM EIF NPM 0-V 3V. 4T- 7TK 7X7 7XB 88E 88G 88J 8AO 8FI 8FJ 8FK 8G5 ABUWG AFKRA ALSLI AZQEC BENPR CCPQU DWQXO FYUFA GHDGH GNUQQ GUQSH K9. M0S M1P M2M M2O M2R MBDVC PADUT PHGZM PHGZT PJZUB PKEHL POGQB PPXIY PQEST PQQKQ PQUKI PRINS PRQQA PSYQQ Q9U 7X8 7QO 8FD FR3 P64 |
| DOI | 10.3758/s13428-013-0365-7 |
| DatabaseName | CrossRef Medline MEDLINE MEDLINE (Ovid) MEDLINE MEDLINE PubMed ProQuest Social Sciences Premium Collection【Remote access available】 ProQuest Central (Corporate) Docstoc Neurosciences Abstracts Health & Medical Collection ProQuest Central (purchase pre-March 2016) Medical Database (Alumni Edition) Psychology Database (Alumni) Social Science Database (Alumni Edition) ProQuest Pharma Collection Hospital Premium Collection Hospital Premium Collection (Alumni Edition) ProQuest Central (Alumni) (purchase pre-March 2016) ProQuest Research Library ProQuest Central (Alumni) ProQuest Central UK/Ireland Social Science Premium Collection ProQuest Central Essentials ProQuest Central ProQuest One Community College ProQuest Central Health Research Premium Collection Health Research Premium Collection (Alumni) ProQuest Central Student ProQuest Research Library ProQuest Health & Medical Complete (Alumni) ProQuest Health & Medical Collection PML(ProQuest Medical Library) Psychology Database Proquest-PRL（Research Library） Social Science Database Research Library (Corporate) Research Library China ProQuest Central Premium ProQuest One Academic ProQuest Health & Medical Research Collection ProQuest One Academic Middle East (New) ProQuest Sociology & Social Sciences Collection ProQuest One Health & Nursing ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Academic ProQuest One Academic UKI Edition ProQuest Central China ProQuest One Social Sciences ProQuest One Psychology ProQuest Central Basic MEDLINE - Academic Biotechnology Research Abstracts Technology Research Database Engineering Research Database Biotechnology and BioEngineering Abstracts |
| DatabaseTitle | CrossRef MEDLINE Medline Complete MEDLINE with Full Text PubMed MEDLINE (Ovid) ProQuest One Psychology Research Library Prep ProQuest Central Student ProQuest One Academic Middle East (New) ProQuest Central Essentials ProQuest Social Science Journals (Alumni Edition) ProQuest Health & Medical Complete (Alumni) ProQuest Central (Alumni Edition) ProQuest One Community College ProQuest One Health & Nursing Research Library (Alumni Edition) ProQuest Pharma Collection Sociology & Social Sciences Collection ProQuest Central China ProQuest Central Health Research Premium Collection Health and Medicine Complete (Alumni Edition) ProQuest Central Korea Health & Medical Research Collection ProQuest Research Library ProQuest Central (New) Research Library China ProQuest Medical Library (Alumni) Social Science Premium Collection ProQuest One Social Sciences ProQuest Central Basic ProQuest One Academic Eastern Edition ProQuest Hospital Collection Health Research Premium Collection (Alumni) ProQuest Psychology Journals (Alumni) Neurosciences Abstracts ProQuest Hospital Collection (Alumni) ProQuest Health & Medical Complete ProQuest Social Science Journals ProQuest Medical Library ProQuest Psychology Journals ProQuest Social Sciences Premium Collection ProQuest One Academic UKI Edition Docstoc ProQuest One Academic ProQuest One Academic (New) ProQuest Central (Alumni) MEDLINE - Academic Engineering Research Database Biotechnology Research Abstracts Technology Research Database Biotechnology and BioEngineering Abstracts |
| DatabaseTitleList | Engineering Research Database ProQuest Health & Medical Complete (Alumni) Engineering Research Database ProQuest One Psychology MEDLINE - Academic MEDLINE |
| Database_xml | – sequence: 1 dbid: NPM name: PubMed url: https://proxy.k.utb.cz/login?url=http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: EIF name: MEDLINE url: https://proxy.k.utb.cz/login?url=https://www.webofscience.com/wos/medline/basic-search sourceTypes: Index Database – sequence: 3 dbid: BENPR name: ProQuest Central url: https://www.proquest.com/central sourceTypes: Aggregation Database |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Psychology |
| EISSN | 1554-3528 |
| EndPage | 130 |
| ExternalDocumentID | 3382920141 23835650 10_3758_s13428_013_0365_7 |
| Genre | Clinical Trial Journal Article |
| GroupedDBID | --- -55 -5G -BR -DZ -EM -ET -~C -~X 0-V 06D 0R~ 0VY 199 1N0 203 23N 2J2 2JN 2JY 2KG 2KM 2LR 2VQ 30V 3V. 4.4 406 408 40E 53G 5GY 7X7 875 88E 8AO 8FI 8FJ 8G5 8TC 8UJ 95. 96X AAAVM AABHQ AACDK AAHNG AAIAL AAJBT AAJKR AAKPC AANZL AARTL AASML AATNV AATVU AAUYE AAWCG AAYIU AAYQN AAYTO AAYZH AAZMS ABAKF ABDZT ABECU ABFTV ABHLI ABIVO ABJNI ABJOX ABJUD ABKCH ABMQK ABNWP ABPLI ABPPZ ABQBU ABQSL ABSXP ABTEG ABTHY ABTKH ABTMW ABUWG ABXPI ACAOD ACBXY ACDTI ACGFS ACHQT ACHSB ACHXU ACIWK ACKIV ACKNC ACMDZ ACMLO ACNCT ACOKC ACPIV ACPRK ACZOJ ADBBV ADHHG ADHIR ADINQ ADKNI ADKPE ADURQ ADYFF ADZKW AEBTG AEFQL AEGAL AEGNC AEJHL AEJRE AEKMD AEMSY AEOHA AEPYU AESKC AETCA AEVLU AEXYK AFBBN AFFNX AFKRA AFQWF AFRAH AFWTZ AFZKB AGAYW AGDGC AGJBK AGMZJ AGQEE AGQMX AGRTI AGWIL AGWZB AGYKE AHAVH AHBYD AHMBA AHSBF AHYZX AIAKS AIGIU AIIXL AILAN AITGF AJBLW AJRNO AJZVZ ALIPV ALMA_UNASSIGNED_HOLDINGS ALSLI AMKLP AMXSW AMYLF AMYQR AOCGG ARALO ARMRJ ASPBG AVWKF AXYYD AYQZM AZFZN AZQEC B-. BAWUL BENPR BGNMA BPHCQ BVXVI C1A CAG CCPQU COF CSCUP DDRTE DIK DNIVK DPUIP DWQXO E3Z EBD EBLON EBS EIOEI EJD EMOBN ESBYG F5P FEDTE FERAY FFXSO FIGPU FINBP FNLPD FRRFC FSGXE FYUFA GGCAI GGRSB GJIRD GNUQQ GNWQR GQ3 GQ6 GQ7 GUQSH H13 HF~ HMCUK HMJXF HRMNR HVGLF HZ~ H~9 IAO IHR IKXTQ INH IPY IRVIT ITC ITM IWAJR J-C JBSCW JZLTJ KOV LLZTM M1P M2M M2O M2R M4Y MVM N2Q N9A NB0 NPVJJ NQJWS NU0 O9- O93 O9G O9J OHT OK1 P2P P9L PADUT PF- PQQKQ PROAC PSQYO PSYQQ PT4 R9I RIG ROL RPV RSV S16 S1Z S27 S3B SBS SBU SCLPG SHX SISQX SJYHP SNE SNPRN SNX SOHCF SOJ SPISZ SRMVM SSLCW SSXJD STPWE SV3 SZN T13 TN5 TR2 TSG TUC TUS U2A U9L UG4 UKHRP UOJIU UPT UTJUX UZXMN VFIZW VXZ W48 WH7 WK8 XJT XOL XSW Z7R Z7S Z7W Z81 Z83 Z88 Z8N Z92 ZMTXR ZOVNA ZUP AAPKM AAYXX ABBRH ABDBE ABFSG ACSTC ADHKG AEZWR AFDZB AFHIU AFOHR AGQPQ AHPBZ AHWEU AIXLP ATHPR AYFIA CITATION PHGZM PHGZT CGR CUY CVF ECM EIF NPM 4T- 7TK 7XB 8FK ABRTQ K9. MBDVC PJZUB PKEHL POGQB PPXIY PQEST PQUKI PRINS PRQQA Q9U 7X8 7QO 8FD FR3 P64 |
| ID | FETCH-LOGICAL-c576t-5a03eb71e8fc32b3f1a0ff7ad058f7502ffbec6d7e28494c0bb4d85251f84b033 |
| IEDL.DBID | 7X7 |
| ISSN | 1554-3528 1554-351X |
| IngestDate | Fri Jul 11 09:42:01 EDT 2025 Thu Jul 10 19:17:29 EDT 2025 Thu Jul 10 17:28:40 EDT 2025 Fri Jul 25 03:44:35 EDT 2025 Fri Jul 25 05:14:35 EDT 2025 Wed Feb 19 01:53:54 EST 2025 Thu Apr 24 22:52:00 EDT 2025 Tue Jul 01 01:05:50 EDT 2025 Fri Feb 21 02:45:32 EST 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 1 |
| Keywords | Crowdsourcing Longitudinal research MTurk Data quality Mechanical Turk Internet research |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c576t-5a03eb71e8fc32b3f1a0ff7ad058f7502ffbec6d7e28494c0bb4d85251f84b033 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 ObjectType-Article-2 ObjectType-Feature-1 content type line 23 |
| OpenAccessLink | https://pure.eur.nl/en/publications/db02dabd-3856-4b22-afd4-54c1d7aa8806 |
| PMID | 23835650 |
| PQID | 1547945151 |
| PQPubID | 976348 |
| PageCount | 19 |
| ParticipantIDs | proquest_miscellaneous_1534819502 proquest_miscellaneous_1520361797 proquest_miscellaneous_1503548287 proquest_journals_2665618049 proquest_journals_1547945151 pubmed_primary_23835650 crossref_primary_10_3758_s13428_013_0365_7 crossref_citationtrail_10_3758_s13428_013_0365_7 springer_journals_10_3758_s13428_013_0365_7 |
| ProviderPackageCode | CITATION AAYXX |
| PublicationCentury | 2000 |
| PublicationDate | 2014-03-01 |
| PublicationDateYYYYMMDD | 2014-03-01 |
| PublicationDate_xml | – month: 03 year: 2014 text: 2014-03-01 day: 01 |
| PublicationDecade | 2010 |
| PublicationPlace | Boston |
| PublicationPlace_xml | – name: Boston – name: United States – name: New York |
| PublicationTitle | Behavior research methods |
| PublicationTitleAbbrev | Behav Res |
| PublicationTitleAlternate | Behav Res Methods |
| PublicationYear | 2014 |
| Publisher | Springer US Springer Nature B.V |
| Publisher_xml | – name: Springer US – name: Springer Nature B.V |
| References | Peer, E., Paolacci, G., Chandler, J., & Mueller, P. (2012). Selectively Recruiting Participants from Amazon Mechanical Turk Using Qualtrics (May 2, 2012). Available at SSRN: http://ssrn.com/abstract=2100631 Shapiro, D. N., Chandler, J. J., & Mueller, P. A. (2013). Using Mechanical Turk to Study Clinical and Subclinical Populations. ShenhavARandDGGreeneJDDivine intuition: Cognitive style influences belief in GodJournal of Experimental Psychology. General201214134232192892410.1037/a0025391 Goodman, J. K., Cryder, C. E., & Cheema, A. (2012). Data Collection in a Flat World: The Strengths and Weaknesses of Mechanical Turk Samples. Journal of Behavioral Decision Making. SuriSWattsDJCooperation and Contagion in Web-Based, Networked Public Goods ExperimentsPLoS One201163e1683630558892141243110.1371/journal.pone.0016836 PaolacciGChandlerJIpeirotisPRunning experiments on Amazon Mechanical TurkJudgment and Decision Making20105411419 SteeleCMAronsonJStereotype threat and the intellectual test performance of African AmericansJournal of Personality and Social Psychology1995695797811747303210.1037/0022-3514.69.5.797 Ipeirotis, P. (2010). Demographics of Mechanical Turk. CeDER-10–01 working paper, New York University. AndersonN. H.Likableness ratings of 555 personality-trait wordsJournal of Personality and Social Psychology196893272 HortonJJRandDGZeckhauserRJThe online laboratory: Conducting experiments in a real labor marketExperimental Economics201143994210.1007/s10683-011-9273-9 FrederickSCognitive reflection and decision makingJournal of Economic Perspectives2005194254210.1257/089533005775196732 SawyerAGDemand artifacts in laboratory experiments in consumer researchJournal of Consumer Research197514203010.1086/208604 CacioppoJ. T.PettyR. E.Feng KaoC.The efficient assessment of need for cognitionJournal of Personality Assessment1984483306307 GoslingSVazireSSrivastavaSJohnOShould we trust web-based studies? A Comparative Analysis of Six Preconceptions About Internet QuestionnairesAmerican Psychologist200459931041499263610.1037/0003-066X.59.2.93 WestRFMeserveRJStanovichKECognitive sophistication does not attenuate the bias blind spotJournal of Personality and Social Psychology201210335065192266335110.1037/a0028857 FaulFErdfelderEBuchnerALangA-GStatistical power analyses using G*Power 3.1: Tests for correlation and regression analysesBehavior Research Methods200941114911601989782310.3758/BRM.41.4.1149 Mueller, P., & Chandler, J. (2012). Emailing Workers Using Python (March 3, 2012). Available at SSRN: http://ssrn.com/abstract=2100601 BodenhausenGVStereotypes as judgmental heuristics: Evidence of circadian variations in discriminationPsychological Science1990131932210.1111/j.1467-9280.1990.tb00226.x Cooper, S., Khatib, F., Treuille, A., Barbero, J., Lee, J., Beenan, M., . . . Foldit Players (2010). Predicting protein structures with a multilayer online game. Nature, 466, 756–760. doi:10.1038/nature09304 BrockTCBeckerLA'Debriefing' and susceptibility to subsequent experimental manipulationsJournal of Experimental Social Psychology196623510.1016/0022-1031(66)90087-4 Amazon Mechanical Turk Requester Tour. (n.d.). Retrieved from https://requester.mturk.com/tour MasonWSuriSConducting behavioral research on Amazon’s Mechanical TurkBehavior Research Methods20124411232171726610.3758/s13428-011-0124-6 DownsJSHolbrookMBShengSCranorLFAre your participants gaming the system? Screening Mechanical Turk workersProceedings of the 28th international conference on Human factors in computing systems2010New YorkACM23992402 RoschECognitive reference pointsCognitive Psychology19757453254710.1016/0010-0285(75)90021-3 BehrendTSharekDMeadeAWiebeEThe viability of crowdsourcing for survey researchBehavior Research Methods20114338008132143774910.3758/s13428-011-0081-0 BuchananTBirnbaumMHPotential of the Internet for personality researchPsychological experiments on the Internet2000San DiegoAcademic Press12114010.1016/B978-012099980-4/50006-X HansenWBToblerNSGrahamJWAttrition in Substance Abuse Prevention ResearchEvaluation Review19901467768510.1177/0193841X9001400608 SilvermanIShulmanADWiesenthalDLEffects of deceiving and debriefing psychological subjects on performance in later experimentsJournal of Personality and Social Psychology197014320321210.1037/h0028852 DanaherKCrandallCSStereotype threat in applied settings re–examinedJournal of Applied Social Psychology20083861639165510.1111/j.1559-1816.2008.00362.x BassoMRBornsteinRALangJMPractice effects on commonly used measures of executive function across twelve monthsThe Clinical Neuropsychologist19991332832921072660010.1076/clin.13.3.283.1743 Lintott, C. J., Schawinski, K., Slosar, A., Land, K., Bamford, S., Thomas, D., . . . Vandenberg, J. (2008). Galaxy Zoo: morphologies derived from visual inspection of galaxies from the Sloan Digital Sky Survey. Monthly Notices of the Royal Astronomical Society, 389(3), 1179-1189 MunsonSAResnickPMynattEFitzpatrickGHudsonSEdwardsKRoddenTPresenting diverse political opinions: How and how muchProceedings of the 28th International Conference on Human Factors in Computing Systems2010New YorkAssociation for Computing Machinery14571466 BuhrmesterMDKwangTGoslingSDAmazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data?Perspectives on Psychological Science201163510.1177/1745691610393980 Chandler, J., Paolacci, G., & Mueller, P. (2013). Risks and rewards of crowdsourcing marketplaces. In P. Michelucci (Ed.) Handbook of Human Computation. New York: Sage. KitturAChiEHSuhBCrowdsourcing user studies with Mechanical TurkProceedings of the ACM conference on human factors in computing systems2008New YorkACM453456 Goldin, G., Darlow, A. (2013). TurkGate (Version 0.4.0) [Software]. Available from, http://gideongoldin.github.com/TurkGate RibislKMWaltonMAMowbrayCTLukeDADavidsonWSBootsmillerBJMinimizing participant attrition in panel studies through the use of effective retention and tracking strategies: Review and recommendationsEvaluation and Program Planning19991912510.1016/0149-7189(95)00037-2 ReipsUDBirnbaumMHThe Web experiment method: Advantages, disadvantages and solutionsPsychological experiments on the Internet2000San DiegoAcademic Press8911410.1016/B978-012099980-4/50005-8 Fiske, S. T., & Taylor, S. E. (1984). Social cognition. New York: Random House GlinskiRJGlinskiBCSlatinGTNonnaivety contamination in conformity experiments: sources, effects, and implications for controlJournal of Personality and Social Psychology19701647848510.1037/h0030073 SprouseJA validation of Amazon Mechanical Turk for the collection of acceptability judgments in linguistic theoryBehavior Research Methods201143115516730484562128710810.3758/s13428-010-0039-7 PaxtonJMUngarLGreeneJDReflection and reasoning in moral judgmentCognitive Science20123611631772204993110.1111/j.1551-6709.2011.01210.x JohnsonJAAscertaining the validity of Web-based personality inventoriesJournal of Research in Personality20053910312910.1016/j.jrp.2004.09.009 Summerville, A., & Chartier, C. R. (2012). Pseudo-dyadic “interaction” on Amazon’s Mechanical Turk. Behavior Research Methods, 1-9. doi:10.3758/s13428-012-0250-9 BerinskyAJHuberGALenzGSEvaluating online labor markets for experimental research: Amazon.com’s Mechanical TurkPolitical Analysis201220335136810.1093/pan/mpr057 KrantzJHDalalRBirnbaumMHValidity of web-based psychological researchPsychological experiments on the Internet2000New YorkAcademic Press356010.1016/B978-012099980-4/50003-4 von AhnLMaurerBMcMillenCAbrahamDBlumMreCAPTCHA: Human-Based Character Recognition via Web Security MeasuresScience20083211465146810.1126/science.1160379 FinucaneMLGullionCMDeveloping a tool for measuring the decision-making competence of older adultsPsychology and Aging201025227129186392054541310.1037/a0019106 PopeDSimonsohnURound numbers as goals: Evidence from baseball, SAT takers, and the labPsychological Science201122171792114846010.1177/0956797610391098 Mata, A., Fiedler, K., Ferreira, M. B., & Almeida, T. (2013). Reasoning about others’ reasoning. Journal of Experimental Social Psychology. SimmonsJNelsonLSimonsohnUFalse-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significantPsychological Science201122135913662200606110.1177/0956797611417632 EllsworthPCGonzalezRHoggMCooperJQuestions and comparisons: Methods of research in social psychologyThe Sage Handbook of Social Psychology2003LondonSage Publications, Ltd.2442 RandDGThe promise of Mechanical Turk: How online labor markets can help theorists run behavioral experimentsJournal of Theoretical Biology20122991721792140208110.1016/j.jtbi.2011.03.004 RosnowRLAikenLSMediation of artifacts in behavioral researchJournal of Experimental Social Psychology19739318120110.1016/0022-1031(73)90009-7 ChiltonLBHortonJJMillerRCAzenkotSBennettPChandrasekarRChickeringMIpeirotisPLawEMityaginAProvostFvon AhnLTask search in a human computation market. In Proceedings of the ACM SIGKDD workshop on human computation (pp. 1–9)HCOMP ’09: Proceedings of the ACM SIGKDD Workshop on Human Computation (77–85)2009New YorkACM DownsJSHolbrookMPeelEScreening Participants on Mechanical Turk: Techniques and Justifications2012VancouverPaper presented at the annual conference of the Association for Consumer ResearchOctober 2012 GaggioliARivaGWorking the CrowdScience200812144310.1126/science.321.5895.1443a OppenheimerDMMeyvisTDavidenkoNInstructional manipulation checks: Detecting satisficing to increase statistical powerJournal of Experimental Social Psychology20094586787210.1016/j.jesp.2009.03.009 EdlundJESagarinBJSkowronskiJJJohnsonSJKutterJWhatever happens in the laboratory stays in the laboratory: The prevalence and prevention of participant crosstalkPersonality and Social Psychology Bulletin2009356356421923429810.1177/0146167208331255 365_CR50 J. T. Cacioppo (365_CR10) 1984; 48 GV Bodenhausen (365_CR6) 1990; 1 PC Ellsworth (365_CR19) 2003 365_CR13 AG Sawyer (365_CR49) 1975; 1 365_CR56 365_CR11 E Rosch (365_CR47) 1975; 7 JS Downs (365_CR16) 2010 365_CR1 G Paolacci (365_CR40) 2010; 5 KM Ribisl (365_CR46) 1999; 19 SA Munson (365_CR38) 2010 J Sprouse (365_CR54) 2011; 43 365_CR42 MR Basso (365_CR3) 1999; 13 T Buchanan (365_CR8) 2000 AJ Berinsky (365_CR5) 2012; 20 T Behrend (365_CR4) 2011; 43 W Mason (365_CR35) 2012; 44 365_CR37 ML Finucane (365_CR21) 2010; 25 S Gosling (365_CR27) 2004; 59 N. H. Anderson (365_CR2) 1968; 9 A Gaggioli (365_CR23) 2008; 12 RJ Glinski (365_CR24) 1970; 16 JM Paxton (365_CR41) 2012; 36 A Shenhav (365_CR51) 2012; 141 S Suri (365_CR57) 2011; 6 365_CR36 D Pope (365_CR43) 2011; 22 S Frederick (365_CR22) 2005; 19 365_CR34 365_CR30 K Danaher (365_CR14) 2008; 38 JA Johnson (365_CR31) 2005; 39 JH Krantz (365_CR33) 2000 LB Chilton (365_CR12) 2009 365_CR26 A Kittur (365_CR32) 2008 CM Steele (365_CR55) 1995; 69 DM Oppenheimer (365_CR39) 2009; 45 DG Rand (365_CR44) 2012; 299 F Faul (365_CR20) 2009; 41 365_CR25 MD Buhrmester (365_CR9) 2011; 6 JS Downs (365_CR15) 2012 J Simmons (365_CR53) 2011; 22 I Silverman (365_CR52) 1970; 14 RF West (365_CR59) 2012; 103 WB Hansen (365_CR28) 1990; 14 365_CR18 L von Ahn (365_CR58) 2008; 321 JJ Horton (365_CR29) 2011; 4 JE Edlund (365_CR17) 2009; 35 RL Rosnow (365_CR48) 1973; 9 TC Brock (365_CR7) 1966; 2 UD Reips (365_CR45) 2000 |
| References_xml | – reference: HansenWBToblerNSGrahamJWAttrition in Substance Abuse Prevention ResearchEvaluation Review19901467768510.1177/0193841X9001400608 – reference: Mueller, P., & Chandler, J. (2012). Emailing Workers Using Python (March 3, 2012). Available at SSRN: http://ssrn.com/abstract=2100601 – reference: Goodman, J. K., Cryder, C. E., & Cheema, A. (2012). Data Collection in a Flat World: The Strengths and Weaknesses of Mechanical Turk Samples. Journal of Behavioral Decision Making. – reference: SprouseJA validation of Amazon Mechanical Turk for the collection of acceptability judgments in linguistic theoryBehavior Research Methods201143115516730484562128710810.3758/s13428-010-0039-7 – reference: AndersonN. H.Likableness ratings of 555 personality-trait wordsJournal of Personality and Social Psychology196893272 – reference: RosnowRLAikenLSMediation of artifacts in behavioral researchJournal of Experimental Social Psychology19739318120110.1016/0022-1031(73)90009-7 – reference: RibislKMWaltonMAMowbrayCTLukeDADavidsonWSBootsmillerBJMinimizing participant attrition in panel studies through the use of effective retention and tracking strategies: Review and recommendationsEvaluation and Program Planning19991912510.1016/0149-7189(95)00037-2 – reference: ChiltonLBHortonJJMillerRCAzenkotSBennettPChandrasekarRChickeringMIpeirotisPLawEMityaginAProvostFvon AhnLTask search in a human computation market. In Proceedings of the ACM SIGKDD workshop on human computation (pp. 1–9)HCOMP ’09: Proceedings of the ACM SIGKDD Workshop on Human Computation (77–85)2009New YorkACM – reference: BuchananTBirnbaumMHPotential of the Internet for personality researchPsychological experiments on the Internet2000San DiegoAcademic Press12114010.1016/B978-012099980-4/50006-X – reference: Shapiro, D. N., Chandler, J. J., & Mueller, P. A. (2013). Using Mechanical Turk to Study Clinical and Subclinical Populations. – reference: SteeleCMAronsonJStereotype threat and the intellectual test performance of African AmericansJournal of Personality and Social Psychology1995695797811747303210.1037/0022-3514.69.5.797 – reference: KitturAChiEHSuhBCrowdsourcing user studies with Mechanical TurkProceedings of the ACM conference on human factors in computing systems2008New YorkACM453456 – reference: ReipsUDBirnbaumMHThe Web experiment method: Advantages, disadvantages and solutionsPsychological experiments on the Internet2000San DiegoAcademic Press8911410.1016/B978-012099980-4/50005-8 – reference: FrederickSCognitive reflection and decision makingJournal of Economic Perspectives2005194254210.1257/089533005775196732 – reference: PaxtonJMUngarLGreeneJDReflection and reasoning in moral judgmentCognitive Science20123611631772204993110.1111/j.1551-6709.2011.01210.x – reference: DownsJSHolbrookMPeelEScreening Participants on Mechanical Turk: Techniques and Justifications2012VancouverPaper presented at the annual conference of the Association for Consumer ResearchOctober 2012 – reference: Amazon Mechanical Turk Requester Tour. (n.d.). Retrieved from https://requester.mturk.com/tour – reference: EllsworthPCGonzalezRHoggMCooperJQuestions and comparisons: Methods of research in social psychologyThe Sage Handbook of Social Psychology2003LondonSage Publications, Ltd.2442 – reference: BuhrmesterMDKwangTGoslingSDAmazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data?Perspectives on Psychological Science201163510.1177/1745691610393980 – reference: BassoMRBornsteinRALangJMPractice effects on commonly used measures of executive function across twelve monthsThe Clinical Neuropsychologist19991332832921072660010.1076/clin.13.3.283.1743 – reference: GaggioliARivaGWorking the CrowdScience200812144310.1126/science.321.5895.1443a – reference: JohnsonJAAscertaining the validity of Web-based personality inventoriesJournal of Research in Personality20053910312910.1016/j.jrp.2004.09.009 – reference: Fiske, S. T., & Taylor, S. E. (1984). Social cognition. New York: Random House – reference: Chandler, J., Paolacci, G., & Mueller, P. (2013). Risks and rewards of crowdsourcing marketplaces. In P. Michelucci (Ed.) Handbook of Human Computation. New York: Sage. – reference: Mata, A., Fiedler, K., Ferreira, M. B., & Almeida, T. (2013). Reasoning about others’ reasoning. Journal of Experimental Social Psychology. – reference: Ipeirotis, P. (2010). Demographics of Mechanical Turk. CeDER-10–01 working paper, New York University. – reference: Lintott, C. J., Schawinski, K., Slosar, A., Land, K., Bamford, S., Thomas, D., . . . Vandenberg, J. (2008). Galaxy Zoo: morphologies derived from visual inspection of galaxies from the Sloan Digital Sky Survey. Monthly Notices of the Royal Astronomical Society, 389(3), 1179-1189 – reference: MasonWSuriSConducting behavioral research on Amazon’s Mechanical TurkBehavior Research Methods20124411232171726610.3758/s13428-011-0124-6 – reference: BerinskyAJHuberGALenzGSEvaluating online labor markets for experimental research: Amazon.com’s Mechanical TurkPolitical Analysis201220335136810.1093/pan/mpr057 – reference: DanaherKCrandallCSStereotype threat in applied settings re–examinedJournal of Applied Social Psychology20083861639165510.1111/j.1559-1816.2008.00362.x – reference: ShenhavARandDGGreeneJDDivine intuition: Cognitive style influences belief in GodJournal of Experimental Psychology. General201214134232192892410.1037/a0025391 – reference: MunsonSAResnickPMynattEFitzpatrickGHudsonSEdwardsKRoddenTPresenting diverse political opinions: How and how muchProceedings of the 28th International Conference on Human Factors in Computing Systems2010New YorkAssociation for Computing Machinery14571466 – reference: FinucaneMLGullionCMDeveloping a tool for measuring the decision-making competence of older adultsPsychology and Aging201025227129186392054541310.1037/a0019106 – reference: HortonJJRandDGZeckhauserRJThe online laboratory: Conducting experiments in a real labor marketExperimental Economics201143994210.1007/s10683-011-9273-9 – reference: SimmonsJNelsonLSimonsohnUFalse-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significantPsychological Science201122135913662200606110.1177/0956797611417632 – reference: BrockTCBeckerLA'Debriefing' and susceptibility to subsequent experimental manipulationsJournal of Experimental Social Psychology196623510.1016/0022-1031(66)90087-4 – reference: DownsJSHolbrookMBShengSCranorLFAre your participants gaming the system? Screening Mechanical Turk workersProceedings of the 28th international conference on Human factors in computing systems2010New YorkACM23992402 – reference: RoschECognitive reference pointsCognitive Psychology19757453254710.1016/0010-0285(75)90021-3 – reference: Summerville, A., & Chartier, C. R. (2012). Pseudo-dyadic “interaction” on Amazon’s Mechanical Turk. Behavior Research Methods, 1-9. doi:10.3758/s13428-012-0250-9 – reference: von AhnLMaurerBMcMillenCAbrahamDBlumMreCAPTCHA: Human-Based Character Recognition via Web Security MeasuresScience20083211465146810.1126/science.1160379 – reference: GoslingSVazireSSrivastavaSJohnOShould we trust web-based studies? A Comparative Analysis of Six Preconceptions About Internet QuestionnairesAmerican Psychologist200459931041499263610.1037/0003-066X.59.2.93 – reference: OppenheimerDMMeyvisTDavidenkoNInstructional manipulation checks: Detecting satisficing to increase statistical powerJournal of Experimental Social Psychology20094586787210.1016/j.jesp.2009.03.009 – reference: WestRFMeserveRJStanovichKECognitive sophistication does not attenuate the bias blind spotJournal of Personality and Social Psychology201210335065192266335110.1037/a0028857 – reference: Peer, E., Paolacci, G., Chandler, J., & Mueller, P. (2012). Selectively Recruiting Participants from Amazon Mechanical Turk Using Qualtrics (May 2, 2012). Available at SSRN: http://ssrn.com/abstract=2100631 – reference: PaolacciGChandlerJIpeirotisPRunning experiments on Amazon Mechanical TurkJudgment and Decision Making20105411419 – reference: Goldin, G., Darlow, A. (2013). TurkGate (Version 0.4.0) [Software]. Available from, http://gideongoldin.github.com/TurkGate/ – reference: Cooper, S., Khatib, F., Treuille, A., Barbero, J., Lee, J., Beenan, M., . . . Foldit Players (2010). Predicting protein structures with a multilayer online game. Nature, 466, 756–760. doi:10.1038/nature09304 – reference: RandDGThe promise of Mechanical Turk: How online labor markets can help theorists run behavioral experimentsJournal of Theoretical Biology20122991721792140208110.1016/j.jtbi.2011.03.004 – reference: BodenhausenGVStereotypes as judgmental heuristics: Evidence of circadian variations in discriminationPsychological Science1990131932210.1111/j.1467-9280.1990.tb00226.x – reference: BehrendTSharekDMeadeAWiebeEThe viability of crowdsourcing for survey researchBehavior Research Methods20114338008132143774910.3758/s13428-011-0081-0 – reference: EdlundJESagarinBJSkowronskiJJJohnsonSJKutterJWhatever happens in the laboratory stays in the laboratory: The prevalence and prevention of participant crosstalkPersonality and Social Psychology Bulletin2009356356421923429810.1177/0146167208331255 – reference: SilvermanIShulmanADWiesenthalDLEffects of deceiving and debriefing psychological subjects on performance in later experimentsJournal of Personality and Social Psychology197014320321210.1037/h0028852 – reference: PopeDSimonsohnURound numbers as goals: Evidence from baseball, SAT takers, and the labPsychological Science201122171792114846010.1177/0956797610391098 – reference: CacioppoJ. T.PettyR. E.Feng KaoC.The efficient assessment of need for cognitionJournal of Personality Assessment1984483306307 – reference: SuriSWattsDJCooperation and Contagion in Web-Based, Networked Public Goods ExperimentsPLoS One201163e1683630558892141243110.1371/journal.pone.0016836 – reference: GlinskiRJGlinskiBCSlatinGTNonnaivety contamination in conformity experiments: sources, effects, and implications for controlJournal of Personality and Social Psychology19701647848510.1037/h0030073 – reference: FaulFErdfelderEBuchnerALangA-GStatistical power analyses using G*Power 3.1: Tests for correlation and regression analysesBehavior Research Methods200941114911601989782310.3758/BRM.41.4.1149 – reference: SawyerAGDemand artifacts in laboratory experiments in consumer researchJournal of Consumer Research197514203010.1086/208604 – reference: KrantzJHDalalRBirnbaumMHValidity of web-based psychological researchPsychological experiments on the Internet2000New YorkAcademic Press356010.1016/B978-012099980-4/50003-4 – volume: 22 start-page: 1359 year: 2011 ident: 365_CR53 publication-title: Psychological Science doi: 10.1177/0956797611417632 – volume: 48 start-page: 306 issue: 3 year: 1984 ident: 365_CR10 publication-title: Journal of Personality Assessment doi: 10.1207/s15327752jpa4803_13 – volume: 45 start-page: 867 year: 2009 ident: 365_CR39 publication-title: Journal of Experimental Social Psychology doi: 10.1016/j.jesp.2009.03.009 – ident: 365_CR18 – volume: 59 start-page: 93 year: 2004 ident: 365_CR27 publication-title: American Psychologist doi: 10.1037/0003-066X.59.2.93 – ident: 365_CR37 doi: 10.2139/ssrn.2100601 – ident: 365_CR11 doi: 10.1007/978-1-4614-8806-4_30 – volume: 19 start-page: 25 issue: 4 year: 2005 ident: 365_CR22 publication-title: Journal of Economic Perspectives doi: 10.1257/089533005775196732 – start-page: 121 volume-title: Psychological experiments on the Internet year: 2000 ident: 365_CR8 doi: 10.1016/B978-012099980-4/50006-X – volume: 5 start-page: 411 year: 2010 ident: 365_CR40 publication-title: Judgment and Decision Making doi: 10.1017/S1930297500002205 – volume: 1 start-page: 319 year: 1990 ident: 365_CR6 publication-title: Psychological Science doi: 10.1111/j.1467-9280.1990.tb00226.x – volume-title: HCOMP ’09: Proceedings of the ACM SIGKDD Workshop on Human Computation (77–85) year: 2009 ident: 365_CR12 doi: 10.1145/1837885.1837889 – start-page: 24 volume-title: The Sage Handbook of Social Psychology year: 2003 ident: 365_CR19 – volume: 12 start-page: 1443 year: 2008 ident: 365_CR23 publication-title: Science doi: 10.1126/science.321.5895.1443a – ident: 365_CR34 doi: 10.1111/j.1365-2966.2008.13689.x – volume: 16 start-page: 478 year: 1970 ident: 365_CR24 publication-title: Journal of Personality and Social Psychology doi: 10.1037/h0030073 – ident: 365_CR56 doi: 10.3758/s13428-012-0250-9 – ident: 365_CR26 doi: 10.1002/bdm.1753 – volume: 44 start-page: 1 issue: 1 year: 2012 ident: 365_CR35 publication-title: Behavior Research Methods doi: 10.3758/s13428-011-0124-6 – ident: 365_CR36 doi: 10.1016/j.jesp.2013.01.010 – ident: 365_CR50 doi: 10.1177/2167702612469015 – volume: 141 start-page: 423 issue: 3 year: 2012 ident: 365_CR51 publication-title: Journal of Experimental Psychology. General doi: 10.1037/a0025391 – volume: 321 start-page: 1465 year: 2008 ident: 365_CR58 publication-title: Science doi: 10.1126/science.1160379 – volume: 13 start-page: 283 issue: 3 year: 1999 ident: 365_CR3 publication-title: The Clinical Neuropsychologist doi: 10.1076/clin.13.3.283.1743 – volume: 1 start-page: 20 issue: 4 year: 1975 ident: 365_CR49 publication-title: Journal of Consumer Research doi: 10.1086/208604 – ident: 365_CR30 – ident: 365_CR1 – volume: 39 start-page: 103 year: 2005 ident: 365_CR31 publication-title: Journal of Research in Personality doi: 10.1016/j.jrp.2004.09.009 – start-page: 1457 volume-title: Proceedings of the 28th International Conference on Human Factors in Computing Systems year: 2010 ident: 365_CR38 doi: 10.1145/1753326.1753543 – volume: 25 start-page: 271 issue: 2 year: 2010 ident: 365_CR21 publication-title: Psychology and Aging doi: 10.1037/a0019106 – volume: 22 start-page: 71 issue: 1 year: 2011 ident: 365_CR43 publication-title: Psychological Science doi: 10.1177/0956797610391098 – volume: 19 start-page: 1 year: 1999 ident: 365_CR46 publication-title: Evaluation and Program Planning doi: 10.1016/0149-7189(95)00037-2 – volume: 7 start-page: 532 issue: 4 year: 1975 ident: 365_CR47 publication-title: Cognitive Psychology doi: 10.1016/0010-0285(75)90021-3 – volume: 103 start-page: 506 issue: 3 year: 2012 ident: 365_CR59 publication-title: Journal of Personality and Social Psychology doi: 10.1037/a0028857 – start-page: 35 volume-title: Psychological experiments on the Internet year: 2000 ident: 365_CR33 doi: 10.1016/B978-012099980-4/50003-4 – volume: 35 start-page: 635 year: 2009 ident: 365_CR17 publication-title: Personality and Social Psychology Bulletin doi: 10.1177/0146167208331255 – volume: 6 start-page: 3 year: 2011 ident: 365_CR9 publication-title: Perspectives on Psychological Science doi: 10.1177/1745691610393980 – volume: 38 start-page: 1639 issue: 6 year: 2008 ident: 365_CR14 publication-title: Journal of Applied Social Psychology doi: 10.1111/j.1559-1816.2008.00362.x – volume: 14 start-page: 203 issue: 3 year: 1970 ident: 365_CR52 publication-title: Journal of Personality and Social Psychology doi: 10.1037/h0028852 – volume: 36 start-page: 163 issue: 1 year: 2012 ident: 365_CR41 publication-title: Cognitive Science doi: 10.1111/j.1551-6709.2011.01210.x – volume: 4 start-page: 399 year: 2011 ident: 365_CR29 publication-title: Experimental Economics doi: 10.1007/s10683-011-9273-9 – volume: 299 start-page: 172 year: 2012 ident: 365_CR44 publication-title: Journal of Theoretical Biology doi: 10.1016/j.jtbi.2011.03.004 – volume: 69 start-page: 797 issue: 5 year: 1995 ident: 365_CR55 publication-title: Journal of Personality and Social Psychology doi: 10.1037/0022-3514.69.5.797 – volume: 41 start-page: 1149 year: 2009 ident: 365_CR20 publication-title: Behavior Research Methods doi: 10.3758/BRM.41.4.1149 – ident: 365_CR13 doi: 10.1038/nature09304 – volume: 20 start-page: 351 issue: 3 year: 2012 ident: 365_CR5 publication-title: Political Analysis doi: 10.1093/pan/mpr057 – volume: 43 start-page: 155 issue: 1 year: 2011 ident: 365_CR54 publication-title: Behavior Research Methods doi: 10.3758/s13428-010-0039-7 – volume-title: Screening Participants on Mechanical Turk: Techniques and Justifications year: 2012 ident: 365_CR15 – volume: 9 start-page: 272 issue: 3 year: 1968 ident: 365_CR2 publication-title: Journal of Personality and Social Psychology doi: 10.1037/h0025907 – start-page: 2399 volume-title: Proceedings of the 28th international conference on Human factors in computing systems year: 2010 ident: 365_CR16 doi: 10.1145/1753326.1753688 – volume: 9 start-page: 181 issue: 3 year: 1973 ident: 365_CR48 publication-title: Journal of Experimental Social Psychology doi: 10.1016/0022-1031(73)90009-7 – volume: 2 start-page: 3 year: 1966 ident: 365_CR7 publication-title: Journal of Experimental Social Psychology doi: 10.1016/0022-1031(66)90087-4 – ident: 365_CR25 – ident: 365_CR42 doi: 10.2139/ssrn.2100631 – volume: 43 start-page: 800 issue: 3 year: 2011 ident: 365_CR4 publication-title: Behavior Research Methods doi: 10.3758/s13428-011-0081-0 – start-page: 89 volume-title: Psychological experiments on the Internet year: 2000 ident: 365_CR45 doi: 10.1016/B978-012099980-4/50005-8 – volume: 14 start-page: 677 year: 1990 ident: 365_CR28 publication-title: Evaluation Review doi: 10.1177/0193841X9001400608 – start-page: 453 volume-title: Proceedings of the ACM conference on human factors in computing systems year: 2008 ident: 365_CR32 doi: 10.1145/1357054.1357127 – volume: 6 start-page: e16836 issue: 3 year: 2011 ident: 365_CR57 publication-title: PLoS One doi: 10.1371/journal.pone.0016836 |
| SSID | ssj0038214 |
| Score | 2.606672 |
| Snippet | Crowdsourcing services—particularly Amazon Mechanical Turk—have made it easy for behavioral scientists to recruit research participants. However, researchers... Crowdsourcing services--particularly Amazon Mechanical Turk--have made it easy for behavioral scientists to recruit research participants. However, researchers... Crowdsourcing services-particularly Amazon Mechanical Turk-have made it easy for behavioral scientists to recruit research participants. However, researchers... |
| SourceID | proquest pubmed crossref springer |
| SourceType | Aggregation Database Index Database Enrichment Source Publisher |
| StartPage | 112 |
| SubjectTerms | Analysis of Variance Behavioral Research - methods Behavioral Science and Psychology Cognitive Psychology Crowdsourcing Crowdsourcing - methods Data Collection - methods Data Collection - standards Experiments Health Literacy Healthy Volunteers Humans Information Dissemination Internet Longitudinal Studies Male Population Surveillance - methods Practice (Psychology) Psychology Research Personnel Research Subjects - psychology Scientists Studies Task Performance and Analysis Video Games - psychology Video Games - statistics & numerical data Websites Workers |
| SummonAdditionalLinks | – databaseName: SpringerLink Journals (ICM) dbid: U2A link: http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlZ1LS8QwEIAHHxcv4tv6IoInpZg26Tb1tiyKCO5pF_ZWkjZBULvi7gr6jzz5I_xjzmwfKuqC56SFTJLJN8wL4Ai1vg1cK_OFdcaXCKV-InM0XBNj49ioME8o3_m627rsy6tBNKjyuEd1tHvtkpxqarIrEWpPR4GQVE2ZuhFQaFY8D4sRmu4Ux9UP27X6FSoMZOm-_P2z7w_QD6r84RGdPjQXK7BcESJrl1u6CnO2WIOlRlE9r8NNd1gU-v3tyY7fX9m0XRBr3-uXYcGuLWXykuBZb_J4yyjsCgHvjHW-RE0zXeSsOXMMsZV9ZuuzqvwPceEG9C_Oe51Lv-qY4GdoN4z9SHNhTRxY5TIRGuECzZ2Ldc4j5ZANQudwz1p5bPFVSmTGjZG5ipBxnJKGC7EJC8WwsNvAcqelMEplPMF77bSyQZIhXLnQ5UloWx7wWoxpVpUTp64WdymaFST5tJR8ipJPSfJp7MFx88lDWUtj1uS9em_S6lqNUuQ91B-IYMGvwwgbiIMKjR4PDpthvC_kBNGFHU7oF1yglYaG4qw55J5FVTVzDqUwJyhTD7bKo9MsCjFIIChzD07qs_RlDX-teOdfs3dhCdlNluFwe7AwfpzYfeSjsTmY3ocP-ecKfg priority: 102 providerName: Springer Nature |
| Title | Nonnaïveté among Amazon Mechanical Turk workers: Consequences and solutions for behavioral researchers |
| URI | https://link.springer.com/article/10.3758/s13428-013-0365-7 https://www.ncbi.nlm.nih.gov/pubmed/23835650 https://www.proquest.com/docview/1547945151 https://www.proquest.com/docview/2665618049 https://www.proquest.com/docview/1503548287 https://www.proquest.com/docview/1520361797 https://www.proquest.com/docview/1534819502 |
| Volume | 46 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV1fT9swED_x54UXNAaDMIaMxBNThBO7jcML6lAZAlGhiUrlKbJjW0hAwmg7aftGe9qH4Itx16SBCehTpNiJ5Lvz3e985zuAXdT6LvLtPBTOm1AiKA1TadFxTY1LEqNim9J95_Ne-6QvTwetQX3gNqzTKqc6caKobZnTGfk-mnoUHbS-0eH9z5C6RlF0tW6hMQ-LVLqMpDoZNA6XUHEkq0imQFi8P4yEpHrM1M-AkruS_23RK4D5Kjg6sTnHH2C5BousU3F3BeZc8RGWGp31exWue2VR6Md_v9zo8S-bdA5inTv9pyzYuaNLvcQDdjl-uGGUgYVY74AdvUigZrqwrBE_hgiWPV_cZ3UlIIKIa9A_7l4enYR184QwRxdiFLY0F84kkVM-F7ERPtLc-0Rb3lIeYULsPbKvbROHBiqVOTdGWtVCuOOVNFyIT7BQlIXbAGa9lsIolfMUt7jXykVpjjjLx96msWsHwKdkzPK6sjg1uLjN0MMgymcV5TOkfEaUz5IA9ppP7quyGrMmb015k9U7bJg9y8Obw4g7EBkq9H8C2GmGcetQPEQXrhzTL7hAhw19xllzKFKLWmvmHLrNnCJNA1ivRKdZFCIigZiZB_B1Kksv1vDeijdnr_gzLCFuk1Uq3BYsjB7G7gtio5HZnmyAbVjsfL866-LzW7d38QPf9uPOEwONEfY |
| linkProvider | ProQuest |
| linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1Lb9QwEB5V5UAviDcpBYwEF1BUJ_ZuHCSEqkK1pd09baW9GTu2VQmalO4uVfuPuMCP6B9jZvNoUWFvPduJNOPxzDeeF8Ar1Po-Cf0iFj7YWCIojXPp0HHNrc8yq1KXU73zcNQfHMjPk95kBX61tTCUVtnqxIWidlVBb-SbaOpRdND6Jh-Ov8c0NYqiq-0IjVos9vzZKbps0_e7H_F8X6fpzqfx9iBupgrEBWLrWdwzXHibJV6FQqRWhMTwEDLjeE8FtJ9pCEhX32UeNXcuC26tdKqHOCAoaTk9gKLKv4WGl5Ozl006B0-oNJF15FQgDN-cJkJS_2ean0DJZNnftu8aoL0WjF3YuJ27cKcBp2yrlqZ7sOLL-7DW6cizB3A4qsrSXPz-4WcXP9liUhHbOjLnVcmGnoqI6czZeH7ylVHGF2LLd2z7SsI2M6VjnbgzRMzsslEAazoPESR9CAc3wtZHsFpWpX8CzAUjhVWq4DmqlGCUT_ICcV1Ig8tT34-At2zURdPJnAZqfNPo0RDndc15jZzXxHmdRfCm--S4buOxbPNGeza6udFTfSl__1xGnINIVKG_FcHLbhmvKsVfTOmrOf2CC3QQ0Uddtociw6gll-6h6ukceRrB41p0OqIQgQnE6DyCt60sXaHhfxSvL6f4BdwejIf7en93tPcU1hAzyjoNbwNWZydz_wxx2cw-X1wGBl9u-vb9AV3-S5Y |
| linkToPdf | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV3daxQxEB_KFaQv4rerVSPoi7JcdpO7zQoite3RWnsUaaFvMdkkFNTd2rtT6n_kk---9h9z5vajleq99TnZhZnMx28ykxmAZ2j1fRKGRSx8sLFEUBrn0mHgmlufZValLqf3zrvj4daBfHc4OFyC3-1bGCqrbG3i3FC7qqA78j66ehQd9L5JPzRlEXsbozfHX2OaIEWZ1nacRi0iO_70O4Zvk9fbG3jWz9N0tLm_vhU3EwbiAnH2NB4YLrzNEq9CIVIrQmJ4CJlxfKAC-tI0BKRx6DKPVjyXBbdWOjVATBCUtJwuQ9H8L2cUFfVg-e3meO9D6weEShNZ51EFgvL-JBGSukHTNAUqLcv-9oSX4O2l1Ozc441uwPUGqrK1WrZuwpIvb8FKZzFPb8PRuCpLc_brm5-e_WTzuUVs7Yv5UZVs19OTYpIAtj87-cSo_guR5iu2fqF8m5nSsU74GeJndt42gDV9iAig3oGDK2HsXeiVVenvA3PBSGGVKniOBiYY5ZO8QJQX0uDy1A8j4C0bddH0NafxGp81xjfEeV1zXiPnNXFeZxG86D45rpt6LNq82p6NbvR7os-l8Z_LiHoQlyqMviJ42i2j4lI2xpS-mtEvuMBwESPWRXsoT4w2c-EeekudI08juFeLTkcU4jGBiJ1H8LKVpQs0_I_iB4spfgLXUPP0--3xzkNYQQAp65q8VehNT2b-EYK0qX3caAODj1etgH8AY4pRMQ |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Nonnaivete+among+Amazon+Mechanical+Turk+workers%3A+Consequences+and+solutions+for+behavioral+researchers&rft.jtitle=Behavior+research+methods&rft.au=Chandler%2C+Jesse&rft.au=Mueller%2C+Pam&rft.au=Paolacci%2C+Gabriele&rft.date=2014-03-01&rft.issn=1554-351X&rft.eissn=1554-3528&rft.volume=46&rft.issue=1&rft.spage=112&rft.epage=130&rft_id=info:doi/10.3758%2Fs13428-013-0365-7&rft.externalDBID=NO_FULL_TEXT |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1554-3528&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1554-3528&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1554-3528&client=summon |