Online panels in social science research: Expanding sampling methods beyond Mechanical Turk
Amazon Mechanical Turk (MTurk) is widely used by behavioral scientists to recruit research participants. MTurk offers advantages over traditional student subject pools, but it also has important limitations. In particular, the MTurk population is small and potentially overused, and some groups of in...
Saved in:
| Published in | Behavior research methods Vol. 51; no. 5; pp. 2022 - 2038 |
|---|---|
| Main Authors | , , , , |
| Format | Journal Article |
| Language | English |
| Published |
New York
Springer US
01.10.2019
Springer Nature B.V |
| Subjects | |
| Online Access | Get full text |
Cover
Loading…
| Abstract | Amazon Mechanical Turk (MTurk) is widely used by behavioral scientists to recruit research participants. MTurk offers advantages over traditional student subject pools, but it also has important limitations. In particular, the MTurk population is small and potentially overused, and some groups of interest to behavioral scientists are underrepresented and difficult to recruit. Here we examined whether online research panels can avoid these limitations. Specifically, we compared sample composition, data quality (measured by effect sizes, internal reliability, and attention checks), and the non-naivete of participants recruited from MTurk and Prime Panels—an aggregate of online research panels. Prime Panels participants were more diverse in age, family composition, religiosity, education, and political attitudes. Prime Panels participants also reported less exposure to classic protocols and produced larger effect sizes, but only after screening out several participants who failed a screening task. We conclude that online research panels offer a unique opportunity for research, yet one with some important trade-offs. |
|---|---|
| AbstractList | Amazon Mechanical Turk (MTurk) is widely used by behavioral scientists to recruit research participants. MTurk offers advantages over traditional student subject pools, but it also has important limitations. In particular, the MTurk population is small and potentially overused, and some groups of interest to behavioral scientists are underrepresented and difficult to recruit. Here we examined whether online research panels can avoid these limitations. Specifically, we compared sample composition, data quality (measured by effect sizes, internal reliability, and attention checks), and the non-naivete of participants recruited from MTurk and Prime Panels—an aggregate of online research panels. Prime Panels participants were more diverse in age, family composition, religiosity, education, and political attitudes. Prime Panels participants also reported less exposure to classic protocols and produced larger effect sizes, but only after screening out several participants who failed a screening task. We conclude that online research panels offer a unique opportunity for research, yet one with some important trade-offs. Amazon Mechanical Turk (MTurk) is widely used by behavioral scientists to recruit research participants. MTurk offers advantages over traditional student subject pools, but it also has important limitations. In particular, the MTurk population is small and potentially overused, and some groups of interest to behavioral scientists are underrepresented and difficult to recruit. Here we examined whether online research panels can avoid these limitations. Specifically, we compared sample composition, data quality (measured by effect sizes, internal reliability, and attention checks), and the non-naivete of participants recruited from MTurk and Prime Panels-an aggregate of online research panels. Prime Panels participants were more diverse in age, family composition, religiosity, education, and political attitudes. Prime Panels participants also reported less exposure to classic protocols and produced larger effect sizes, but only after screening out several participants who failed a screening task. We conclude that online research panels offer a unique opportunity for research, yet one with some important trade-offs.Amazon Mechanical Turk (MTurk) is widely used by behavioral scientists to recruit research participants. MTurk offers advantages over traditional student subject pools, but it also has important limitations. In particular, the MTurk population is small and potentially overused, and some groups of interest to behavioral scientists are underrepresented and difficult to recruit. Here we examined whether online research panels can avoid these limitations. Specifically, we compared sample composition, data quality (measured by effect sizes, internal reliability, and attention checks), and the non-naivete of participants recruited from MTurk and Prime Panels-an aggregate of online research panels. Prime Panels participants were more diverse in age, family composition, religiosity, education, and political attitudes. Prime Panels participants also reported less exposure to classic protocols and produced larger effect sizes, but only after screening out several participants who failed a screening task. We conclude that online research panels offer a unique opportunity for research, yet one with some important trade-offs. |
| Author | Robinson, Jonathan Rosenzweig, Cheskie Litman, Leib Moss, Aaron J. Chandler, Jesse |
| Author_xml | – sequence: 1 givenname: Jesse surname: Chandler fullname: Chandler, Jesse organization: University of Michigan – sequence: 2 givenname: Cheskie surname: Rosenzweig fullname: Rosenzweig, Cheskie organization: Columbia University – sequence: 3 givenname: Aaron J. surname: Moss fullname: Moss, Aaron J. organization: Prime Research Solutions – sequence: 4 givenname: Jonathan surname: Robinson fullname: Robinson, Jonathan organization: Lander College – sequence: 5 givenname: Leib surname: Litman fullname: Litman, Leib email: leib.litman@touro.edu organization: Lander College |
| BackLink | https://www.ncbi.nlm.nih.gov/pubmed/31512174$$D View this record in MEDLINE/PubMed |
| BookMark | eNp9kUtv3CAUhVGVKq_mD3RRIXWTjVMeBuwuKlVR2kRKlE26ygJhfD1DimEKdpX8-zKdvBdZIK7Ed44O9-yhrRADIPSRkiOuRPMlU16zpiK0LYcpXql3aJcKUVdcsGbr2byD9nK-IYQ3jNbbaIdTQRlV9S66vgzeBcArE8Bn7ALO0TrjcbYOggWcIINJdvkVn9wWqHdhgbMZV349jDAtY59xB3cx9PgC7NIEZ4v8ak6_P6D3g_EZDu7vffTrx8nV8Wl1fvnz7Pj7eWUlUVMl664X7UBsp6S1wPoapLCiJoI30shOGtVRMfBOUMVlCxSUlIazoWNWtjXh--jbxnc1dyP0FsKUjNer5EaT7nQ0Tr98CW6pF_GvlqpVsm2LweG9QYp_ZsiTHl224H1ZSpyzZqxphVJM8IJ-foXexDmF8j3NOJGC08IW6tPzRI9RHvZegGYD2BRzTjBo6yYzubgO6LymRK8r1puKdalY_69YqyJlr6QP7m-K-EaUCxwWkJ5iv6H6B1w6uVA |
| CitedBy_id | crossref_primary_10_2139_ssrn_4514749 crossref_primary_10_1177_0003122420971805 crossref_primary_10_1177_13684302211040107 crossref_primary_10_2196_35797 crossref_primary_10_1016_j_jand_2023_11_021 crossref_primary_10_1038_s41562_021_01057_0 crossref_primary_10_1177_1069031X221096637 crossref_primary_10_20982_tqmp_16_4_p334 crossref_primary_10_1177_10870547221112948 crossref_primary_10_1016_j_paid_2021_111139 crossref_primary_10_1177_14613557211064047 crossref_primary_10_1080_10508619_2022_2148062 crossref_primary_10_1016_j_techsoc_2024_102464 crossref_primary_10_1007_s13347_022_00511_9 crossref_primary_10_3390_su15107980 crossref_primary_10_3389_fpubh_2025_1502006 crossref_primary_10_1080_21565503_2024_2430523 crossref_primary_10_3389_fpsyg_2024_1385410 crossref_primary_10_1080_09658211_2023_2198264 crossref_primary_10_1016_j_chbah_2024_100097 crossref_primary_10_1080_0361073X_2023_2168441 crossref_primary_10_1016_j_chb_2021_106939 crossref_primary_10_1016_j_ejpoleco_2022_102249 crossref_primary_10_3390_socsci12030133 crossref_primary_10_1016_j_im_2025_104134 crossref_primary_10_3389_frdem_2024_1451310 crossref_primary_10_1177_08901171221075612 crossref_primary_10_1038_s41467_022_30073_5 crossref_primary_10_1002_csr_2874 crossref_primary_10_1086_723017 crossref_primary_10_1038_s41598_022_21972_0 crossref_primary_10_1007_s11292_022_09546_6 crossref_primary_10_2196_46421 crossref_primary_10_1108_CCIJ_04_2023_0050 crossref_primary_10_1177_00222429231213011 crossref_primary_10_1016_j_jagp_2020_07_013 crossref_primary_10_1016_j_jebo_2023_11_017 crossref_primary_10_1108_IJEFM_11_2022_0089 crossref_primary_10_30658_jicrcr_5_1_1 crossref_primary_10_1177_13684302221118534 crossref_primary_10_3390_su132313458 crossref_primary_10_1093_alcalc_agac005 crossref_primary_10_1177_00187208251317470 crossref_primary_10_1515_humor_2021_0137 crossref_primary_10_1007_s11199_020_01212_3 crossref_primary_10_1016_j_sleh_2021_10_004 crossref_primary_10_1177_07395329241298967 crossref_primary_10_3390_bs10090133 crossref_primary_10_1080_17457289_2025_2474423 crossref_primary_10_1108_JPBM_03_2022_3899 crossref_primary_10_1177_01650254211039025 crossref_primary_10_1016_j_pursup_2024_100922 crossref_primary_10_1086_722345 crossref_primary_10_3390_rel13090791 crossref_primary_10_1016_j_chb_2024_108405 crossref_primary_10_1080_1553118X_2025_2462087 crossref_primary_10_2139_ssrn_4558486 crossref_primary_10_2196_26994 crossref_primary_10_1002_rhc3_12219 crossref_primary_10_1177_01461672241289833 crossref_primary_10_1016_j_healthpol_2021_05_003 crossref_primary_10_1177_0275074020941676 crossref_primary_10_1186_s13034_025_00872_x crossref_primary_10_1177_2333721420960259 crossref_primary_10_1016_j_jesp_2023_104489 crossref_primary_10_1001_jamanetworkopen_2024_12880 crossref_primary_10_1016_j_psychres_2021_114144 crossref_primary_10_1176_appi_ps_20220168 crossref_primary_10_1109_TTS_2021_3120317 crossref_primary_10_1007_s11199_023_01377_7 crossref_primary_10_1177_01925121241293459 crossref_primary_10_3390_buildings12122073 crossref_primary_10_1016_j_janxdis_2020_102290 crossref_primary_10_1007_s10551_021_04950_w crossref_primary_10_1016_j_ijintrel_2021_03_008 crossref_primary_10_1007_s11109_020_09620_7 crossref_primary_10_1016_j_jesp_2023_104575 crossref_primary_10_1587_transinf_2023EDP7145 crossref_primary_10_1145_3533047 crossref_primary_10_1080_17439760_2024_2378709 crossref_primary_10_1080_00224499_2023_2291091 crossref_primary_10_1001_jamanetworkopen_2024_60465 crossref_primary_10_3390_ijerph21081052 crossref_primary_10_1038_s41598_023_46048_5 crossref_primary_10_1371_journal_pone_0268121 crossref_primary_10_1016_j_eneco_2024_107716 crossref_primary_10_3389_fgwh_2023_1095112 crossref_primary_10_3390_adolescents2040035 crossref_primary_10_3390_bs14090794 crossref_primary_10_1177_00222437221147221 crossref_primary_10_1093_geronb_gbac105 crossref_primary_10_1111_jbl_70012 crossref_primary_10_1080_1051712X_2022_2121508 crossref_primary_10_1001_jamapediatrics_2020_3864 crossref_primary_10_2139_ssrn_3978470 crossref_primary_10_1007_s10869_023_09890_y crossref_primary_10_1111_jmft_12747 crossref_primary_10_1017_S0033291723003392 crossref_primary_10_1080_23322039_2023_2190214 crossref_primary_10_1016_j_jocrd_2020_100605 crossref_primary_10_1080_19317611_2022_2053027 crossref_primary_10_1177_00027642241301659 crossref_primary_10_1177_08982643211049079 crossref_primary_10_3390_su14148900 crossref_primary_10_1161_CIRCOUTCOMES_124_011069 crossref_primary_10_1089_cyber_2022_0376 crossref_primary_10_1177_00302228221090754 crossref_primary_10_2196_19461 crossref_primary_10_1016_j_tjnut_2023_08_019 crossref_primary_10_1080_13504851_2022_2107982 crossref_primary_10_1007_s10804_021_09381_5 crossref_primary_10_1080_07418825_2024_2361719 crossref_primary_10_1007_s10869_024_09989_w crossref_primary_10_1108_IJBM_11_2022_0496 crossref_primary_10_1080_03637751_2022_2125995 crossref_primary_10_1016_j_drugpo_2020_103053 crossref_primary_10_1007_s00146_024_02121_9 crossref_primary_10_1016_j_ijhcs_2025_103445 crossref_primary_10_1111_bjso_12417 crossref_primary_10_1080_03623319_2021_1969882 crossref_primary_10_1177_13684302221147005 crossref_primary_10_3389_frcha_2023_1148273 crossref_primary_10_1177_09567976241236162 crossref_primary_10_1016_j_jand_2022_10_007 crossref_primary_10_1371_journal_pone_0312361 crossref_primary_10_1080_09658211_2023_2198326 crossref_primary_10_1080_07317115_2025_2454977 crossref_primary_10_1177_0149206320969787 crossref_primary_10_1080_13825585_2023_2249189 crossref_primary_10_1371_journal_pone_0244534 crossref_primary_10_1016_j_cdnut_2023_102036 crossref_primary_10_1177_26350106251326509 crossref_primary_10_2196_63032 crossref_primary_10_2139_ssrn_3827984 crossref_primary_10_1057_s41300_024_00220_9 crossref_primary_10_1177_08944393211006847 crossref_primary_10_1093_ijpor_edac022 crossref_primary_10_1007_s10508_025_03105_6 crossref_primary_10_1177_00219096241287710 crossref_primary_10_1016_j_chbr_2021_100145 crossref_primary_10_1038_s41598_023_28569_1 crossref_primary_10_1080_10888691_2024_2442567 crossref_primary_10_1016_j_csl_2021_101269 crossref_primary_10_1016_j_jth_2024_101980 crossref_primary_10_1007_s10447_024_09585_w crossref_primary_10_1007_s10676_021_09607_w crossref_primary_10_1111_sltb_12654 crossref_primary_10_1080_03623319_2020_1852856 crossref_primary_10_1016_j_bandc_2021_105789 crossref_primary_10_1007_s41811_021_00114_w crossref_primary_10_1080_10454446_2024_2377542 crossref_primary_10_1287_isre_2022_0305 crossref_primary_10_1016_j_sleepe_2022_100051 crossref_primary_10_1017_jmo_2021_1 crossref_primary_10_1080_0267257X_2021_2001028 crossref_primary_10_1016_j_rmal_2024_100152 crossref_primary_10_3390_ijerph20095727 crossref_primary_10_1016_j_jadohealth_2023_03_007 crossref_primary_10_1073_pnas_2204529119 crossref_primary_10_1145_3635145 crossref_primary_10_1038_s41598_022_07508_6 crossref_primary_10_1002_acp_4106 crossref_primary_10_1016_j_jclepro_2024_141449 crossref_primary_10_1007_s40615_022_01350_2 crossref_primary_10_1001_jamanetworkopen_2024_61495 crossref_primary_10_1016_j_addbeh_2021_107155 crossref_primary_10_1371_journal_pone_0312392 crossref_primary_10_1016_j_anzjph_2023_100102 crossref_primary_10_1007_s12525_023_00656_1 crossref_primary_10_3758_s13428_021_01694_3 crossref_primary_10_1093_jopart_muaa019 crossref_primary_10_1002_joec_12243 crossref_primary_10_1139_cjfr_2020_0274 crossref_primary_10_1007_s11469_024_01381_2 crossref_primary_10_1080_17439760_2021_1975157 crossref_primary_10_1016_j_tra_2021_10_016 crossref_primary_10_1080_10447318_2023_2212219 crossref_primary_10_3758_s13428_022_01944_y crossref_primary_10_1016_j_ijintrel_2020_04_005 crossref_primary_10_2139_ssrn_3758620 crossref_primary_10_1080_00380237_2024_2330470 crossref_primary_10_1007_s12144_021_02216_2 crossref_primary_10_1016_j_jbusres_2021_09_035 crossref_primary_10_1080_1553118X_2025_2471527 crossref_primary_10_1080_10503307_2023_2241983 crossref_primary_10_1108_IJPDLM_01_2024_0037 crossref_primary_10_1108_JHTT_08_2021_0241 crossref_primary_10_1016_j_spc_2023_05_030 crossref_primary_10_1016_j_jand_2022_05_009 crossref_primary_10_1371_journal_pone_0255209 crossref_primary_10_2147_OPTH_S356895 crossref_primary_10_1007_s00146_021_01268_z crossref_primary_10_1007_s00146_023_01832_9 crossref_primary_10_3390_rel13060519 crossref_primary_10_5993_AJHB_46_5_1 crossref_primary_10_1371_journal_pone_0278677 crossref_primary_10_1287_isre_2021_1079 crossref_primary_10_1080_00223891_2024_2444448 crossref_primary_10_1108_MD_10_2020_1356 crossref_primary_10_1027_2512_8442_a000080 crossref_primary_10_1016_j_jneb_2024_07_011 crossref_primary_10_3389_frsps_2023_1302029 crossref_primary_10_1186_s41235_024_00577_3 crossref_primary_10_1111_csp2_12647 crossref_primary_10_1109_JIOT_2023_3342092 crossref_primary_10_1257_app_20210428 crossref_primary_10_1111_joms_12929 crossref_primary_10_1177_23337214221082763 crossref_primary_10_1108_TG_07_2021_0118 crossref_primary_10_3390_su14052849 crossref_primary_10_3758_s13428_022_02005_0 crossref_primary_10_1089_neu_2023_0032 crossref_primary_10_1177_25152459221140388 crossref_primary_10_1016_j_pmedr_2024_102776 crossref_primary_10_2478_popets_2022_0002 crossref_primary_10_1080_17439760_2023_2208095 crossref_primary_10_3389_fpubh_2021_642874 crossref_primary_10_1007_s11199_023_01383_9 crossref_primary_10_1080_23311908_2022_2062879 crossref_primary_10_1108_MIP_06_2023_0258 crossref_primary_10_1108_JCM_01_2021_4394 crossref_primary_10_2139_ssrn_3368813 crossref_primary_10_3390_vaccines12080853 crossref_primary_10_1111_bjop_12679 crossref_primary_10_1007_s10508_022_02329_0 crossref_primary_10_1080_13825585_2024_2371177 crossref_primary_10_1108_JCM_04_2022_5302 crossref_primary_10_1177_20563051221103841 crossref_primary_10_1016_j_paid_2023_112212 crossref_primary_10_1111_puar_13823 crossref_primary_10_1111_jasp_13079 crossref_primary_10_1016_j_intell_2024_101830 crossref_primary_10_1108_IMR_07_2021_0231 crossref_primary_10_1038_s41598_021_00752_2 crossref_primary_10_1080_0361073X_2023_2176629 crossref_primary_10_1177_23780231231182908 crossref_primary_10_2139_ssrn_4470068 crossref_primary_10_1016_j_spinee_2024_07_009 crossref_primary_10_2139_ssrn_3765448 crossref_primary_10_3390_su132212393 crossref_primary_10_1136_bmjqs_2020_011801 crossref_primary_10_1177_19485506221082737 crossref_primary_10_1016_j_jpubeco_2024_105284 crossref_primary_10_1080_07481756_2024_2397947 crossref_primary_10_1097_BRS_0000000000005163 crossref_primary_10_1177_0886260520948144 crossref_primary_10_1007_s11199_023_01416_3 crossref_primary_10_1038_s41598_024_56055_9 crossref_primary_10_1371_journal_pone_0279720 crossref_primary_10_1080_23311975_2025_2453624 crossref_primary_10_1177_10664807221123560 crossref_primary_10_1108_IJCHM_07_2020_0696 crossref_primary_10_1167_jov_21_8_19 crossref_primary_10_1080_15378020_2025_2466873 crossref_primary_10_1177_2378023120942449 crossref_primary_10_1371_journal_pone_0313626 crossref_primary_10_1097_NNR_0000000000000671 crossref_primary_10_2478_jos_2022_0035 crossref_primary_10_1016_j_paid_2022_112025 crossref_primary_10_1016_j_socec_2023_102126 crossref_primary_10_1038_s41598_024_57841_1 crossref_primary_10_1016_j_tele_2024_102197 crossref_primary_10_1080_00224499_2024_2352555 crossref_primary_10_3389_fpsyg_2021_648609 crossref_primary_10_1177_0261927X221081010 crossref_primary_10_1016_j_foodqual_2020_104070 crossref_primary_10_1108_JBIM_04_2021_0211 crossref_primary_10_1080_09669582_2022_2149760 crossref_primary_10_1002_acp_70041 crossref_primary_10_1016_j_socscimed_2023_115712 crossref_primary_10_1016_j_pmedr_2023_102194 crossref_primary_10_1186_s12889_025_21291_9 crossref_primary_10_1093_geront_gnac186 crossref_primary_10_1177_02697580221080006 crossref_primary_10_1016_j_chbr_2024_100426 crossref_primary_10_1177_00472875231209983 crossref_primary_10_1111_bjso_12614 crossref_primary_10_3390_ijerph17197143 crossref_primary_10_1016_j_caeai_2024_100243 crossref_primary_10_1371_journal_pone_0277869 crossref_primary_10_1007_s11292_025_09660_1 crossref_primary_10_3390_fi15050164 crossref_primary_10_1016_j_socec_2022_101971 crossref_primary_10_1136_medethics_2020_106524 crossref_primary_10_1007_s13412_020_00660_3 crossref_primary_10_1080_13825585_2024_2360226 crossref_primary_10_1371_journal_pone_0244974 crossref_primary_10_1080_17439760_2023_2208075 crossref_primary_10_1016_j_paid_2023_112340 crossref_primary_10_1097_HJH_0000000000003974 crossref_primary_10_15195_v7_a17 crossref_primary_10_3168_jds_2021_21344 crossref_primary_10_1177_00472875211025097 crossref_primary_10_1177_17480485241305316 crossref_primary_10_1016_j_techfore_2024_123877 crossref_primary_10_1177_0272989X221118078 crossref_primary_10_1016_j_pubrev_2023_102292 crossref_primary_10_2139_ssrn_3742677 crossref_primary_10_1016_j_jeoa_2022_100420 crossref_primary_10_1093_joc_jqac021 crossref_primary_10_1016_j_chbah_2023_100011 crossref_primary_10_1016_j_techsoc_2024_102732 crossref_primary_10_1016_j_jaclp_2021_12_010 crossref_primary_10_1016_j_leaqua_2023_101735 crossref_primary_10_1186_s12966_021_01178_9 crossref_primary_10_1016_j_ssresearch_2023_102947 crossref_primary_10_1007_s10339_024_01250_9 crossref_primary_10_2139_ssrn_3959751 crossref_primary_10_1016_j_techsoc_2021_101806 crossref_primary_10_1080_15213269_2024_2354789 crossref_primary_10_1007_s10869_024_10000_9 crossref_primary_10_1080_17457289_2021_1924749 crossref_primary_10_1002_asi_24712 crossref_primary_10_1093_cybsec_tyad002 crossref_primary_10_1007_s10865_022_00351_0 crossref_primary_10_1007_s42972_021_00035_0 crossref_primary_10_1111_jcpp_13570 crossref_primary_10_2139_ssrn_2811636 crossref_primary_10_1016_j_archger_2020_104247 crossref_primary_10_1016_j_jbef_2023_100790 crossref_primary_10_1016_j_ejpoleco_2024_102554 crossref_primary_10_1371_journal_pone_0260380 crossref_primary_10_1016_j_compedu_2023_104925 crossref_primary_10_1093_pnasnexus_pgae469 crossref_primary_10_1111_cfs_12989 crossref_primary_10_1080_02699931_2021_1964940 crossref_primary_10_1371_journal_pone_0287837 crossref_primary_10_1080_19419899_2023_2181706 crossref_primary_10_1016_j_jhealeco_2025_102968 crossref_primary_10_3727_152599521X16192004803593 crossref_primary_10_1080_1068316X_2024_2364286 crossref_primary_10_1093_jcmc_zmac008 crossref_primary_10_1177_10575677231199236 crossref_primary_10_1080_00224499_2022_2130856 crossref_primary_10_1016_j_vaccine_2022_11_014 crossref_primary_10_1080_08853134_2020_1858844 crossref_primary_10_1080_16066359_2024_2415590 crossref_primary_10_1016_j_jretai_2022_04_002 crossref_primary_10_1016_j_metip_2020_100032 crossref_primary_10_1177_00811750241312226 crossref_primary_10_1098_rsos_230447 crossref_primary_10_1111_joms_12983 crossref_primary_10_2139_ssrn_4855851 crossref_primary_10_1177_13684302211059699 crossref_primary_10_1016_j_jesp_2021_104182 crossref_primary_10_1177_21568693211043156 crossref_primary_10_3390_vaccines11020279 crossref_primary_10_1016_j_jesp_2022_104401 crossref_primary_10_1515_ajle_2023_0125 crossref_primary_10_1080_0361073X_2023_2233366 crossref_primary_10_1002_asi_24724 crossref_primary_10_1016_j_jand_2024_11_009 crossref_primary_10_1007_s10508_024_02905_6 crossref_primary_10_2196_38589 crossref_primary_10_1016_j_teler_2024_100128 crossref_primary_10_2139_ssrn_3861906 crossref_primary_10_1007_s10211_022_00401_5 crossref_primary_10_1177_15562646211060997 crossref_primary_10_1007_s40806_022_00321_4 crossref_primary_10_1086_728051 crossref_primary_10_1186_s12871_024_02547_0 crossref_primary_10_1177_0306624X221124849 crossref_primary_10_2196_28897 crossref_primary_10_1016_j_jpain_2022_10_012 crossref_primary_10_1007_s12369_023_01003_2 crossref_primary_10_3390_ijerph19169967 crossref_primary_10_1093_milmed_usac127 crossref_primary_10_1371_journal_pone_0274966 crossref_primary_10_1111_scs_13115 crossref_primary_10_1016_j_chbah_2024_100052 crossref_primary_10_2139_ssrn_4124980 crossref_primary_10_1177_13684302221085832 crossref_primary_10_1080_00913367_2022_2079026 crossref_primary_10_2139_ssrn_4602741 crossref_primary_10_1080_08974438_2024_2447947 crossref_primary_10_1002_jcad_12369 crossref_primary_10_1177_00110000241301110 crossref_primary_10_7717_peerj_cs_296 crossref_primary_10_1080_25742442_2021_2007718 crossref_primary_10_1001_jamanetworkopen_2022_0376 crossref_primary_10_3390_su14042199 crossref_primary_10_2196_64724 crossref_primary_10_3389_fpsyg_2020_589390 crossref_primary_10_1002_mar_22058 crossref_primary_10_1016_j_leaqua_2023_101722 crossref_primary_10_1093_ntr_ntac078 crossref_primary_10_1002_cpp_2732 crossref_primary_10_1080_10410236_2022_2031452 crossref_primary_10_1007_s10896_023_00582_y crossref_primary_10_1080_10447318_2024_2384821 crossref_primary_10_1111_jcpp_13427 crossref_primary_10_1177_17456916211004899 crossref_primary_10_1016_j_poetic_2023_101861 crossref_primary_10_5334_joc_259 crossref_primary_10_1177_00302228241278770 crossref_primary_10_1108_EJM_08_2022_0583 crossref_primary_10_1007_s11409_024_09391_0 crossref_primary_10_1016_j_teler_2024_100133 crossref_primary_10_1177_02762374241248117 |
| Cites_doi | 10.1017/XPS.2015.19 10.31234/osf.io/jq589 10.1177/1948550617703168 10.1177/0956797615585115 10.1177/2053168015604648 10.1037/0022-3514.51.3.515 10.1038/466029a 10.3758/s13428-013-0434-y 10.3758/s13421-011-0104-1 10.1016/j.jesp.2018.07.012 10.1146/annurev-clinpsy-021815-093623 10.1093/pan/mpr057 10.1016/j.jesp.2015.11.004 10.2308/accr-51447 10.1177/01461672952111004 10.1177/2167702612469015 10.1177/0963721414531598 10.1016/j.jesp.2017.01.006 10.3758/s13428-017-0963-x 10.1017/xps.2014.7 10.1037/pag0000285 10.1038/ncomms4677 10.1111/j.1468-0017.2006.00297.x 10.1177/2158244016636433 10.1177/2053168018822174 10.1016/j.chb.2017.08.038 10.3758/s13428-015-0578-z 10.2196/jmir.2903 10.1177/2053168015622072 10.1007/s10683-011-9273-9 10.1002/bdm.1753 10.2501/JAR-41-5-63-71 10.1146/annurev-psych-010814-015321 10.1002/9781118763520.ch10 10.1017/S1930297500005611 10.1017/S0140525X10000300 10.1145/3159652.3159661 10.5840/monist197659224 10.1093/jcr/ucx047 10.1073/pnas.1819310116 10.1177/1745691610393980 10.1177/2158244017712774 10.1073/pnas.1808083115 10.1257/089533005775196732 10.1037/pspa0000105 10.1016/j.tics.2017.06.007 10.1126/science.7455683 10.1080/00913367.2016.1269304 10.3758/s13423-017-1348-y 10.1002/9781118763520.ch2 10.3886/ICPSR36390.v1 10.3758/s13428-014-0483-x |
| ContentType | Journal Article |
| Copyright | The Author(s) 2019 Copyright Springer Nature B.V. 2019 |
| Copyright_xml | – notice: The Author(s) 2019 – notice: Copyright Springer Nature B.V. 2019 |
| DBID | C6C AAYXX CITATION CGR CUY CVF ECM EIF NPM 4T- 7TK K9. 7X8 5PM |
| DOI | 10.3758/s13428-019-01273-7 |
| DatabaseName | Springer Nature OA Free Journals CrossRef Medline MEDLINE MEDLINE (Ovid) MEDLINE MEDLINE PubMed Docstoc Neurosciences Abstracts ProQuest Health & Medical Complete (Alumni) MEDLINE - Academic PubMed Central (Full Participant titles) |
| DatabaseTitle | CrossRef MEDLINE Medline Complete MEDLINE with Full Text PubMed MEDLINE (Ovid) ProQuest Health & Medical Complete (Alumni) Neurosciences Abstracts Docstoc MEDLINE - Academic |
| DatabaseTitleList | ProQuest Health & Medical Complete (Alumni) MEDLINE MEDLINE - Academic CrossRef |
| Database_xml | – sequence: 1 dbid: C6C name: Springer Nature OA Free Journals url: http://www.springeropen.com/ sourceTypes: Publisher – sequence: 2 dbid: NPM name: PubMed url: https://proxy.k.utb.cz/login?url=http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 3 dbid: EIF name: MEDLINE url: https://proxy.k.utb.cz/login?url=https://www.webofscience.com/wos/medline/basic-search sourceTypes: Index Database |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Psychology |
| EISSN | 1554-3528 |
| EndPage | 2038 |
| ExternalDocumentID | PMC6797699 31512174 10_3758_s13428_019_01273_7 |
| Genre | Journal Article |
| GroupedDBID | --- -55 -5G -BR -DZ -EM -ET -~C -~X 0-V 06D 0R~ 0VY 199 1N0 203 23N 2J2 2JN 2JY 2KG 2KM 2LR 2VQ 30V 3V. 4.4 406 408 40E 53G 5GY 7X7 875 88E 8AO 8FI 8FJ 8G5 8TC 8UJ 95. 96X AAAVM AABHQ AACDK AAHNG AAIAL AAJBT AAJKR AAKPC AANZL AARTL AASML AATNV AATVU AAUYE AAWCG AAYIU AAYQN AAYTO AAYZH AAZMS ABAKF ABDZT ABECU ABFTV ABHLI ABIVO ABJNI ABJOX ABJUD ABKCH ABMQK ABNWP ABPLI ABPPZ ABQBU ABQSL ABSXP ABTEG ABTHY ABTKH ABTMW ABUWG ABXPI ACAOD ACBXY ACDTI ACGFS ACHQT ACHSB ACHXU ACIWK ACKIV ACKNC ACMDZ ACMLO ACNCT ACOKC ACPIV ACPRK ACZOJ ADBBV ADHHG ADHIR ADINQ ADKNI ADKPE ADURQ ADYFF ADZKW AEBTG AEFQL AEGAL AEGNC AEJHL AEJRE AEKMD AEMSY AEOHA AEPYU AESKC AETCA AEVLU AEXYK AFBBN AFFNX AFKRA AFQWF AFRAH AFWTZ AFZKB AGAYW AGDGC AGJBK AGMZJ AGQEE AGQMX AGRTI AGWIL AGWZB AGYKE AHAVH AHBYD AHMBA AHSBF AHYZX AIAKS AIGIU AIIXL AILAN AITGF AJBLW AJRNO AJZVZ ALIPV ALMA_UNASSIGNED_HOLDINGS ALSLI AMKLP AMXSW AMYLF AMYQR AOCGG ARALO ARMRJ ASPBG AVWKF AXYYD AYQZM AZFZN AZQEC B-. BAWUL BENPR BGNMA BPHCQ BVXVI C1A C6C CAG CCPQU COF CSCUP DDRTE DIK DNIVK DPUIP DWQXO E3Z EBD EBLON EBS EIOEI EJD EMOBN ESBYG F5P FEDTE FERAY FFXSO FIGPU FINBP FNLPD FRRFC FSGXE FYUFA GGCAI GGRSB GJIRD GNUQQ GNWQR GQ3 GQ6 GQ7 GUQSH H13 HF~ HMCUK HMJXF HRMNR HVGLF HZ~ H~9 IAO IHR IKXTQ INH IPY IRVIT ITC ITM IWAJR J-C JBSCW JZLTJ KOV LLZTM M1P M2M M2O M2R M4Y MVM N2Q N9A NB0 NPVJJ NQJWS NU0 O9- O93 O9G O9J OHT OK1 P2P P9L PADUT PF- PQQKQ PROAC PSQYO PSYQQ PT4 R9I RIG ROL RPV RSV S16 S1Z S27 S3B SBS SBU SCLPG SHX SISQX SJYHP SNE SNPRN SNX SOHCF SOJ SPISZ SRMVM SSLCW SSXJD STPWE SV3 SZN T13 TN5 TR2 TSG TUC TUS U2A U9L UG4 UKHRP UOJIU UPT UTJUX UZXMN VFIZW VXZ W48 WH7 WK8 XJT XOL XSW Z7R Z7S Z7W Z81 Z83 Z88 Z8N Z92 ZMTXR ZOVNA ZUP AAPKM AAYXX ABBRH ABDBE ABFSG ACSTC ADHKG AEZWR AFDZB AFHIU AFOHR AGQPQ AHPBZ AHWEU AIXLP ATHPR AYFIA CITATION PHGZM PHGZT CGR CUY CVF ECM EIF NPM 4T- 7TK ABRTQ K9. 7X8 5PM |
| ID | FETCH-LOGICAL-c607t-64bd59f0cb76cce2d4e65c5405386a6b6a7b15f3b517369e1e766a32fb2c69403 |
| IEDL.DBID | C6C |
| ISSN | 1554-3528 1554-351X |
| IngestDate | Thu Aug 21 17:54:54 EDT 2025 Tue Aug 05 10:15:03 EDT 2025 Sun Jul 13 04:27:39 EDT 2025 Thu Apr 03 07:03:23 EDT 2025 Thu Apr 24 23:02:34 EDT 2025 Tue Jul 01 01:05:55 EDT 2025 Fri Feb 21 02:37:00 EST 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 5 |
| Keywords | Data collection Prime panels Mechanical Turk Online experimentation |
| Language | English |
| License | Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c607t-64bd59f0cb76cce2d4e65c5405386a6b6a7b15f3b517369e1e766a32fb2c69403 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 content type line 23 |
| OpenAccessLink | https://doi.org/10.3758/s13428-019-01273-7 |
| PMID | 31512174 |
| PQID | 2306531289 |
| PQPubID | 976348 |
| PageCount | 17 |
| ParticipantIDs | pubmedcentral_primary_oai_pubmedcentral_nih_gov_6797699 proquest_miscellaneous_2289577253 proquest_journals_2306531289 pubmed_primary_31512174 crossref_citationtrail_10_3758_s13428_019_01273_7 crossref_primary_10_3758_s13428_019_01273_7 springer_journals_10_3758_s13428_019_01273_7 |
| ProviderPackageCode | CITATION AAYXX |
| PublicationCentury | 2000 |
| PublicationDate | 2019-10-01 |
| PublicationDateYYYYMMDD | 2019-10-01 |
| PublicationDate_xml | – month: 10 year: 2019 text: 2019-10-01 day: 01 |
| PublicationDecade | 2010 |
| PublicationPlace | New York |
| PublicationPlace_xml | – name: New York – name: United States |
| PublicationTitle | Behavior research methods |
| PublicationTitleAbbrev | Behav Res |
| PublicationTitleAlternate | Behav Res Methods |
| PublicationYear | 2019 |
| Publisher | Springer US Springer Nature B.V |
| Publisher_xml | – name: Springer US – name: Springer Nature B.V |
| References | DavidaiSWhy do Americans believe in economic mobility? Economic inequality, external attributions of wealth and poverty, and the belief in economic mobilityJournal of Experimental Social Psychology20187913814810.1016/j.jesp.2018.07.012 PaolacciGChandlerJInside the Turk: Understanding Mechanical Turk as a participant poolCurrent Directions in Psychological Science20142318418810.1177/0963721414531598 TverskyAKahnemanDThe framing of decisions and the psychology of choiceScience198121145345810.1126/science.7455683 LevayKEFreeseJDruckmanJNThe demographic and political composition of Mechanical Turk samplesSage Open2016611710.1177/2158244016636433 HauserDJSchwarzNAttentive Turkers: MTurk participants perform better on online attention checks than do subject pool participantsBehavior Research Methods20164840040710.3758/s13428-015-0578-z25761395 GoslingSDSandyCJJohnOPPotterJWired but not WEIRD: The promise of the internet in reaching more diverse samplesBehavioral and Brain Sciences201033949510.1017/S0140525X1000030020546648 ShapiroDNChandlerJMuellerPAUsing Mechanical Turk to study clinical populationsClinical Psychological Science2013121322010.1177/2167702612469015 JohnOPSrivastavaSPervinLAJohnOPThe Big Five trait taxonomy: History, measurement, and theoretical perspectivesHandbook of personality: Theory and research19992New York, NYGuilford Press102138 HuffCTingleyD“Who are these people?” Evaluating the demographic characteristics and political preferences of MTurk survey respondentsResearch & Politics20152311210.1177/2053168015604648 Pew Research Center. (2016a). Research in the crowdsourcing age, a case study. Retrieved from http://www.pewinternet.org/2016/07/11/research-in-the-crowdsourcing-age-a-case-study FrederickSCognitive reflection and decision makingJournal of Economic Perspectives200519254210.1257/089533005775196732 Robinson, J., Rosenzweig, C., Moss, A. J., & Litman, L. (2019, June 7). Tapped Out or Barely Tapped? Recommendations for How to Harness the Vast and Largely Unused Potential of the Mechanical Turk Participant Pool. PsychArxiv. https://doi.org/10.31234/osf.io/jq589 HillygusD. SunshineJacksonNatalieYoungMcKenzieProfessional respondents in nonprobability online panelsOnline Panel Research2014Chichester, UKJohn Wiley & Sons, Ltd21923710.1002/9781118763520.ch10 PeerEBrandimarteLSamatSAcquistiABeyond the Turk: Alternative platforms for crowdsourcing behavioral researchJournal of Experimental Social Psychology20177015316310.1016/j.jesp.2017.01.006 Flavin, P. (2011). Public attitudes about political equality. Retrieved from http://www.tessexperiments.org/data/flavin235.html ZwaanRAPecherDPaolacciGBouwmeesterSVerkoeijenPDijkstraKZeelenbergRParticipant Nonnaiveté and the reproducibility of cognitive psychologyPsychonomic Bulletin & Review2017251968197210.3758/s13423-017-1348-y GoodmanJKPaolacciGCrowdsourcing consumer researchJournal of Consumer Research20174419621010.1093/jcr/ucx047 HortonJJRandDGZeckhauserRJThe online laboratory: Conducting experiments in a real labor marketExperimental Economics20111439942510.1007/s10683-011-9273-9 ThomsonJJKilling, letting die, and the trolley problemThe Monist19765920421710.5840/monist19765922411662247 HauserMCushmanFYoungLKang-Xing JinRMikhailJA dissociation between moral judgments and justificationsMind & Language20072212110.1111/j.1468-0017.2006.00297.x Ballew, M. T., Goldberg, M. H., Rosenthal, S. A., Gustafson, A., & Leiserowitz, A. (2019). Systems thinking as a pathway to global warming beliefs and attitudes through an ecological worldview. Proceedings of the National Academy of Sciences, 116(17), 8214-8219. ChandlerJShapiroDConducting clinical research using crowdsourced convenience samplesAnnual Review of Clinical Psychology201612538110.1146/annurev-clinpsy-021815-09362326772208 Rivera, E. (2015). IBISWorld industry report 54191: Market research in the US. Retrieved February, 2017, from IBISWorld database LitmanLRobinsonJRosenzweigCThe relationship between motivation, monetary compensation, and data quality among US and India based workers on Mechanical TurkBehavior Research Methods20154751952810.3758/s13428-014-0483-x24907001 Waggoner, B. J. (2018). Atheism and the effects of mortality salience and limited analytic processing ability on religious faith and teleological thinking (Unpublished master’s thesis). Florida International University, Miami, FL. Retrieved from https://etd.ohiolink.edu/!etd.send_file?accession=csu1530473668176033&disposition=inline JacowitzKEKahnemanDMeasures of anchoring in estimation tasksPersonality and Social Psychology Bulletin1995211161116610.1177/01461672952111004 KeesJBerryCBurtonSSheehanKAn analysis of data quality: Professional panels, student subject pools, and Amazon’s Mechanical TurkJournal of Advertising20174614115510.1080/00913367.2016.1269304 CaseyLSChandlerJLevineASProctorAStrolovitchDZIntertemporal differences among MTurk workers: Time-based sample variations and implications for online data collectionSAGE Open20177211510.1177/2158244017712774 PeerEVosgerauJAcquistiAReputation as a sufficient condition for data quality on Amazon Mechanical TurkBehavior Research Methods2014461023103110.3758/s13428-013-0434-y24356996 RandDGPeysakhovichAKraft-ToddGTNewmanGEWurzbacherONowakMAGreeneJDSocial heuristics shape intuitive cooperationNature Communications20145467711210.1038/ncomms4677 American National Election Studies, Stanford University, & University of Michigan. (2016). American National Election Study: 2016 pilot study. Ann Arbor, MI: Inter-University Consortium for Political and Social Research. Retrieved from https://doi.org/10.3886/ICPSR36390.v1 CliffordSJewellRMWaggonerPDAre samples drawn from Mechanical Turk valid for research on political ideology?Research & Politics2015241910.1177/2053168015622072 FootPThe problem of abortion and the doctrine of the double effectVirtues and vices: And other essays in moral philosophy1978Oxford, UKBlackwell1932(Original work published 1967) BerinskyAJHuberGALenzGSEvaluating online labor markets for experimental research: Amazon.com’s Mechanical TurkPolitical Analysis20122035136810.1093/pan/mpr057 FarrellAMGrenierJHLeibyJScoundrels or stars? Theory and evidence on the quality of workers in online labor marketsAccounting Review2017929311410.2308/accr-51447 Pew Research Center. (2017). In America, does more education equal less religion? Retrieved from http://www.pewforum.org/2017/04/26/in-america-does-more-education-equal-less-religion WoodDHarmsPDLowmanGHDeSimoneJAResponse speed and response consistency as mutually validating indicators of data quality in online samplesSocial Psychological and Personality Science2017845446410.1177/1948550617703168 JohnOPNaumannLPSotoCJJohnOPRobinsRWPervinLAParadigm shift to the integrative Big Five trait taxonomy: History, measurement, and conceptual issuesHandbook of personality: Theory and research20083New York, NYGuilford Press114158 CallegaroMVillarAYeagerDKrosnickJCallegaroMA critical review of studies investigating the quality of data obtained with online panels based on probability and nonprobability samplesOnline panel research: A data quality perspective2014Hoboken, NJWiley235310.1002/9781118763520.ch2 GoslingSDMasonWInternet research in psychologyAnnual Review of Psychology20156687790210.1146/annurev-psych-010814-01532125251483 StewartNChandlerJPaolacciGCrowdsourcing samples in cognitive scienceTrends in Cognitive Sciences20172173674810.1016/j.tics.2017.06.00728803699 CraigBMHaysRDPickardASCellaDRevickiDAReeveBBComparison of US panel vendors for online surveysJournal of Medical Internet Research201315e26010.2196/jmir.2903242921593869084 BuhrmesterMKwangTGoslingSDAmazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data?Perspectives on Psychological Science201163510.1177/174569161039398026162106 ChandlerJPaolacciGPeerEMuellerPRatliffKAUsing nonnaive participants can reduce effect sizesPsychological Science2015261131113910.1177/095679761558511526063440 DeriSDavidaiSGilovichTHome alone: Why people believe others’ social lives are richer than their ownJournal of Personality and Social Psychology201711385887710.1037/pspa000010529189037 JonesWLSonnerBSJust say no to traditional student samplesJournal of Advertising Research200141637110.2501/JAR-41-5-63-71 MullinixKJLeeperTJDruckmanJNFreeseJThe generalizability of survey experimentsJournal of Experimental Political Science2015210913810.1017/XPS.2015.19 GoodmanJKCryderCECheemaAData collection in a flat world: The strengths and weaknesses of Mechanical Turk samplesJournal of Behavioral Decision Making20132621322410.1002/bdm.1753 Job, V., Sieber, V., Rothermund, K., & Nikitin, J. (2018). Age differences in implicit theories about willpower: Why older people endorse a nonlimited theory. Psychology and aging, 33(6), 940. DeVoeSEHouseJReplications with MTurkers who are naïve versus experienced with academic studies: A comment on Connors, Khamitov, Moroz, Campbell, and Henderson (2015)Journal of Experimental Social Psychology201667656710.1016/j.jesp.2015.11.004 CoppockALeeperTJMullinixKJGeneralizability of heterogeneous treatment effect estimates across samplesProceedings of the National Academy of Sciences2018115124411244610.1073/pnas.1808083115 DifallahDFilatovaEIpeirotisPDemographics and dynamics of Mechanical Turk WorkersProceedings of the Eleventh ACM International Conference on Web Search and Data Mining2018New York, NYACM Press13514310.1145/3159652.3159661 SurveyMonkey. (2017). Survey respondents in our panel. Retrieved November 17, 2017, from https://www.surveymonkey.com/mp/audience/our-survey-respondents HeenMSLiebermanJDMietheTDA comparison of different online sampling approaches for generating national samplesCenter for Crime and Justice Policy, CCJP2014118 StewartNUngemachCHarrisAJBartelsDMNewellBRPaolacciGChandlerJThe average laboratory samples a population of 7,300 Amazon Mechanical Turk workersJudgment and Decision Making201510479491 ToplakMEWestRFStanovichKEThe Cognitive Reflection Test as a predictor of performance on heu S Frederick (1273_CR22) 2005; 19 L Litman (1273_CR42) 2015; 47 A Tversky (1273_CR61) 1981; 211 C Huff (1273_CR33) 2015; 2 SD Gosling (1273_CR26) 2010; 33 MS Heen (1273_CR29) 2014; 1 M Hauser (1273_CR27) 2007; 22 E Peer (1273_CR45) 2017; 70 1273_CR47 WL Jones (1273_CR38) 2001; 41 1273_CR48 J Kees (1273_CR39) 2017; 46 1273_CR49 N Stewart (1273_CR56) 2015; 10 KA Thomas (1273_CR58) 2017; 77 OP John (1273_CR37) 1999 Y Krupnikov (1273_CR40) 2014; 1 M Callegaro (1273_CR6) 2014 J Chandler (1273_CR9) 2016; 12 SE DeVoe (1273_CR17) 2016; 67 RA Zwaan (1273_CR64) 2017; 25 D Difallah (1273_CR18) 2018 KE Levay (1273_CR41) 2016; 6 S Clifford (1273_CR10) 2015; 2 DN Shapiro (1273_CR54) 2013; 1 1273_CR35 ME Toplak (1273_CR60) 2011; 39 1273_CR62 OP John (1273_CR36) 2008 J Chandler (1273_CR8) 2015; 26 1273_CR20 SD Gosling (1273_CR25) 2015; 66 1273_CR1 S Deri (1273_CR16) 2017; 113 1273_CR2 D. Sunshine Hillygus (1273_CR31) 2014 D Wood (1273_CR63) 2017; 8 M Buhrmester (1273_CR5) 2011; 6 A Coppock (1273_CR12) 2018; 115 P Foot (1273_CR21) 1978 KE Jacowitz (1273_CR34) 1995; 21 AJ Berinsky (1273_CR3) 2012; 20 DO Sears (1273_CR53) 1986; 51 1273_CR51 1273_CR52 N Stewart (1273_CR55) 2017; 21 JJ Thomson (1273_CR59) 1976; 59 DG Rand (1273_CR50) 2014; 5 JK Goodman (1273_CR23) 2013; 26 S Davidai (1273_CR15) 2018; 79 A Coppock (1273_CR13) 2019; 6 JK Goodman (1273_CR24) 2017; 44 M Bialek (1273_CR4) 2018; 50 G Paolacci (1273_CR44) 2014; 23 J Henrich (1273_CR30) 2010; 466 E Peer (1273_CR46) 2014; 46 JJ Horton (1273_CR32) 2011; 14 DJ Hauser (1273_CR28) 2016; 48 KJ Mullinix (1273_CR43) 2015; 2 AM Farrell (1273_CR19) 2017; 92 1273_CR11 LS Casey (1273_CR7) 2017; 7 BM Craig (1273_CR14) 2013; 15 1273_CR57 |
| References_xml | – reference: PeerEBrandimarteLSamatSAcquistiABeyond the Turk: Alternative platforms for crowdsourcing behavioral researchJournal of Experimental Social Psychology20177015316310.1016/j.jesp.2017.01.006 – reference: FootPThe problem of abortion and the doctrine of the double effectVirtues and vices: And other essays in moral philosophy1978Oxford, UKBlackwell1932(Original work published 1967) – reference: WoodDHarmsPDLowmanGHDeSimoneJAResponse speed and response consistency as mutually validating indicators of data quality in online samplesSocial Psychological and Personality Science2017845446410.1177/1948550617703168 – reference: KrupnikovYLevineASCross-sample comparisons and external validityJournal of Experimental Political Science20141598010.1017/xps.2014.7 – reference: ThomsonJJKilling, letting die, and the trolley problemThe Monist19765920421710.5840/monist19765922411662247 – reference: FarrellAMGrenierJHLeibyJScoundrels or stars? Theory and evidence on the quality of workers in online labor marketsAccounting Review2017929311410.2308/accr-51447 – reference: Job, V., Sieber, V., Rothermund, K., & Nikitin, J. (2018). Age differences in implicit theories about willpower: Why older people endorse a nonlimited theory. Psychology and aging, 33(6), 940. – reference: HeenMSLiebermanJDMietheTDA comparison of different online sampling approaches for generating national samplesCenter for Crime and Justice Policy, CCJP2014118 – reference: ChandlerJShapiroDConducting clinical research using crowdsourced convenience samplesAnnual Review of Clinical Psychology201612538110.1146/annurev-clinpsy-021815-09362326772208 – reference: Flavin, P. (2011). Public attitudes about political equality. Retrieved from http://www.tessexperiments.org/data/flavin235.html – reference: JonesWLSonnerBSJust say no to traditional student samplesJournal of Advertising Research200141637110.2501/JAR-41-5-63-71 – reference: GoslingSDSandyCJJohnOPPotterJWired but not WEIRD: The promise of the internet in reaching more diverse samplesBehavioral and Brain Sciences201033949510.1017/S0140525X1000030020546648 – reference: Pew Research Center. (2017). In America, does more education equal less religion? Retrieved from http://www.pewforum.org/2017/04/26/in-america-does-more-education-equal-less-religion/ – reference: BialekMPennycookGThe cognitive reflection test is robust to multiple exposuresBehavior Research Methods2018501953195910.3758/s13428-017-0963-x28849403 – reference: Waggoner, B. J. (2018). Atheism and the effects of mortality salience and limited analytic processing ability on religious faith and teleological thinking (Unpublished master’s thesis). Florida International University, Miami, FL. Retrieved from https://etd.ohiolink.edu/!etd.send_file?accession=csu1530473668176033&disposition=inline – reference: ZwaanRAPecherDPaolacciGBouwmeesterSVerkoeijenPDijkstraKZeelenbergRParticipant Nonnaiveté and the reproducibility of cognitive psychologyPsychonomic Bulletin & Review2017251968197210.3758/s13423-017-1348-y – reference: RandDGPeysakhovichAKraft-ToddGTNewmanGEWurzbacherONowakMAGreeneJDSocial heuristics shape intuitive cooperationNature Communications20145467711210.1038/ncomms4677 – reference: CliffordSJewellRMWaggonerPDAre samples drawn from Mechanical Turk valid for research on political ideology?Research & Politics2015241910.1177/2053168015622072 – reference: KeesJBerryCBurtonSSheehanKAn analysis of data quality: Professional panels, student subject pools, and Amazon’s Mechanical TurkJournal of Advertising20174614115510.1080/00913367.2016.1269304 – reference: LevayKEFreeseJDruckmanJNThe demographic and political composition of Mechanical Turk samplesSage Open2016611710.1177/2158244016636433 – reference: CallegaroMVillarAYeagerDKrosnickJCallegaroMA critical review of studies investigating the quality of data obtained with online panels based on probability and nonprobability samplesOnline panel research: A data quality perspective2014Hoboken, NJWiley235310.1002/9781118763520.ch2 – reference: CraigBMHaysRDPickardASCellaDRevickiDAReeveBBComparison of US panel vendors for online surveysJournal of Medical Internet Research201315e26010.2196/jmir.2903242921593869084 – reference: HuffCTingleyD“Who are these people?” Evaluating the demographic characteristics and political preferences of MTurk survey respondentsResearch & Politics20152311210.1177/2053168015604648 – reference: GoslingSDMasonWInternet research in psychologyAnnual Review of Psychology20156687790210.1146/annurev-psych-010814-01532125251483 – reference: HillygusD. SunshineJacksonNatalieYoungMcKenzieProfessional respondents in nonprobability online panelsOnline Panel Research2014Chichester, UKJohn Wiley & Sons, Ltd21923710.1002/9781118763520.ch10 – reference: Converse, B. A., Epley, N. (2007). With God on our side. Retrieved from http://www.tessexperiments.org/data/converse561.html – reference: JohnOPSrivastavaSPervinLAJohnOPThe Big Five trait taxonomy: History, measurement, and theoretical perspectivesHandbook of personality: Theory and research19992New York, NYGuilford Press102138 – reference: Rivera, E. (2015). IBISWorld industry report 54191: Market research in the US. Retrieved February, 2017, from IBISWorld database – reference: ChandlerJPaolacciGPeerEMuellerPRatliffKAUsing nonnaive participants can reduce effect sizesPsychological Science2015261131113910.1177/095679761558511526063440 – reference: HauserMCushmanFYoungLKang-Xing JinRMikhailJA dissociation between moral judgments and justificationsMind & Language20072212110.1111/j.1468-0017.2006.00297.x – reference: ThomasKACliffordSValidity and Mechanical Turk: An assessment of exclusion methods and interactive experimentsComputers in Human Behavior20177718419710.1016/j.chb.2017.08.038 – reference: HauserDJSchwarzNAttentive Turkers: MTurk participants perform better on online attention checks than do subject pool participantsBehavior Research Methods20164840040710.3758/s13428-015-0578-z25761395 – reference: CaseyLSChandlerJLevineASProctorAStrolovitchDZIntertemporal differences among MTurk workers: Time-based sample variations and implications for online data collectionSAGE Open20177211510.1177/2158244017712774 – reference: DavidaiSWhy do Americans believe in economic mobility? Economic inequality, external attributions of wealth and poverty, and the belief in economic mobilityJournal of Experimental Social Psychology20187913814810.1016/j.jesp.2018.07.012 – reference: SearsDOCollege sophomores in the laboratory: Influences of a narrow data base on social psychology's view of human natureJournal of Personality and Social Psychology198651351553010.1037/0022-3514.51.3.515 – reference: DifallahDFilatovaEIpeirotisPDemographics and dynamics of Mechanical Turk WorkersProceedings of the Eleventh ACM International Conference on Web Search and Data Mining2018New York, NYACM Press13514310.1145/3159652.3159661 – reference: JohnOPNaumannLPSotoCJJohnOPRobinsRWPervinLAParadigm shift to the integrative Big Five trait taxonomy: History, measurement, and conceptual issuesHandbook of personality: Theory and research20083New York, NYGuilford Press114158 – reference: Pew Research Center. (2016a). Research in the crowdsourcing age, a case study. Retrieved from http://www.pewinternet.org/2016/07/11/research-in-the-crowdsourcing-age-a-case-study – reference: BerinskyAJHuberGALenzGSEvaluating online labor markets for experimental research: Amazon.com’s Mechanical TurkPolitical Analysis20122035136810.1093/pan/mpr057 – reference: CoppockALeeperTJMullinixKJGeneralizability of heterogeneous treatment effect estimates across samplesProceedings of the National Academy of Sciences2018115124411244610.1073/pnas.1808083115 – reference: CoppockAMcClellanOAValidating the demographic, political, psychological, and experimental results obtained from a new source of online survey respondentsResearch & Politics20196111410.1177/2053168018822174 – reference: DeVoeSEHouseJReplications with MTurkers who are naïve versus experienced with academic studies: A comment on Connors, Khamitov, Moroz, Campbell, and Henderson (2015)Journal of Experimental Social Psychology201667656710.1016/j.jesp.2015.11.004 – reference: PeerEVosgerauJAcquistiAReputation as a sufficient condition for data quality on Amazon Mechanical TurkBehavior Research Methods2014461023103110.3758/s13428-013-0434-y24356996 – reference: GoodmanJKCryderCECheemaAData collection in a flat world: The strengths and weaknesses of Mechanical Turk samplesJournal of Behavioral Decision Making20132621322410.1002/bdm.1753 – reference: ToplakMEWestRFStanovichKEThe Cognitive Reflection Test as a predictor of performance on heuristics-and-biases tasksMemory & Cognition2011391275128910.3758/s13421-011-0104-1 – reference: BuhrmesterMKwangTGoslingSDAmazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data?Perspectives on Psychological Science201163510.1177/174569161039398026162106 – reference: Robinson, J., Rosenzweig, C., Moss, A. J., & Litman, L. (2019, June 7). Tapped Out or Barely Tapped? Recommendations for How to Harness the Vast and Largely Unused Potential of the Mechanical Turk Participant Pool. PsychArxiv. https://doi.org/10.31234/osf.io/jq589 – reference: Ballew, M. T., Goldberg, M. H., Rosenthal, S. A., Gustafson, A., & Leiserowitz, A. (2019). Systems thinking as a pathway to global warming beliefs and attitudes through an ecological worldview. Proceedings of the National Academy of Sciences, 116(17), 8214-8219. – reference: JacowitzKEKahnemanDMeasures of anchoring in estimation tasksPersonality and Social Psychology Bulletin1995211161116610.1177/01461672952111004 – reference: American National Election Studies, Stanford University, & University of Michigan. (2016). American National Election Study: 2016 pilot study. Ann Arbor, MI: Inter-University Consortium for Political and Social Research. Retrieved from https://doi.org/10.3886/ICPSR36390.v1 – reference: StewartNUngemachCHarrisAJBartelsDMNewellBRPaolacciGChandlerJThe average laboratory samples a population of 7,300 Amazon Mechanical Turk workersJudgment and Decision Making201510479491 – reference: ShapiroDNChandlerJMuellerPAUsing Mechanical Turk to study clinical populationsClinical Psychological Science2013121322010.1177/2167702612469015 – reference: TverskyAKahnemanDThe framing of decisions and the psychology of choiceScience198121145345810.1126/science.7455683 – reference: GoodmanJKPaolacciGCrowdsourcing consumer researchJournal of Consumer Research20174419621010.1093/jcr/ucx047 – reference: SurveyMonkey. (2017). Survey respondents in our panel. Retrieved November 17, 2017, from https://www.surveymonkey.com/mp/audience/our-survey-respondents – reference: HortonJJRandDGZeckhauserRJThe online laboratory: Conducting experiments in a real labor marketExperimental Economics20111439942510.1007/s10683-011-9273-9 – reference: StewartNChandlerJPaolacciGCrowdsourcing samples in cognitive scienceTrends in Cognitive Sciences20172173674810.1016/j.tics.2017.06.00728803699 – reference: HenrichJHeineSJNorenzayanAMost people are not WEIRDNature20104662910.1038/466029a20595995 – reference: Pew Research Center. (2016b). A wider ideological gap between more and less educated adults. Retrieved from http://www.people-press.org/2016/04/26/a-wider-ideological-gap-between-more-and-less-educated-adults/ – reference: LitmanLRobinsonJRosenzweigCThe relationship between motivation, monetary compensation, and data quality among US and India based workers on Mechanical TurkBehavior Research Methods20154751952810.3758/s13428-014-0483-x24907001 – reference: MullinixKJLeeperTJDruckmanJNFreeseJThe generalizability of survey experimentsJournal of Experimental Political Science2015210913810.1017/XPS.2015.19 – reference: PaolacciGChandlerJInside the Turk: Understanding Mechanical Turk as a participant poolCurrent Directions in Psychological Science20142318418810.1177/0963721414531598 – reference: DeriSDavidaiSGilovichTHome alone: Why people believe others’ social lives are richer than their ownJournal of Personality and Social Psychology201711385887710.1037/pspa000010529189037 – reference: FrederickSCognitive reflection and decision makingJournal of Economic Perspectives200519254210.1257/089533005775196732 – volume: 2 start-page: 109 year: 2015 ident: 1273_CR43 publication-title: Journal of Experimental Political Science doi: 10.1017/XPS.2015.19 – ident: 1273_CR52 doi: 10.31234/osf.io/jq589 – volume: 8 start-page: 454 year: 2017 ident: 1273_CR63 publication-title: Social Psychological and Personality Science doi: 10.1177/1948550617703168 – volume: 26 start-page: 1131 year: 2015 ident: 1273_CR8 publication-title: Psychological Science doi: 10.1177/0956797615585115 – volume: 2 start-page: 1 issue: 3 year: 2015 ident: 1273_CR33 publication-title: Research & Politics doi: 10.1177/2053168015604648 – volume: 51 start-page: 515 issue: 3 year: 1986 ident: 1273_CR53 publication-title: Journal of Personality and Social Psychology doi: 10.1037/0022-3514.51.3.515 – volume: 466 start-page: 29 year: 2010 ident: 1273_CR30 publication-title: Nature doi: 10.1038/466029a – volume: 46 start-page: 1023 year: 2014 ident: 1273_CR46 publication-title: Behavior Research Methods doi: 10.3758/s13428-013-0434-y – volume: 39 start-page: 1275 year: 2011 ident: 1273_CR60 publication-title: Memory & Cognition doi: 10.3758/s13421-011-0104-1 – volume: 79 start-page: 138 year: 2018 ident: 1273_CR15 publication-title: Journal of Experimental Social Psychology doi: 10.1016/j.jesp.2018.07.012 – volume: 12 start-page: 53 year: 2016 ident: 1273_CR9 publication-title: Annual Review of Clinical Psychology doi: 10.1146/annurev-clinpsy-021815-093623 – volume: 20 start-page: 351 year: 2012 ident: 1273_CR3 publication-title: Political Analysis doi: 10.1093/pan/mpr057 – volume: 67 start-page: 65 year: 2016 ident: 1273_CR17 publication-title: Journal of Experimental Social Psychology doi: 10.1016/j.jesp.2015.11.004 – ident: 1273_CR51 – volume: 92 start-page: 93 year: 2017 ident: 1273_CR19 publication-title: Accounting Review doi: 10.2308/accr-51447 – volume: 1 start-page: 1 year: 2014 ident: 1273_CR29 publication-title: Center for Crime and Justice Policy, CCJP – volume: 21 start-page: 1161 year: 1995 ident: 1273_CR34 publication-title: Personality and Social Psychology Bulletin doi: 10.1177/01461672952111004 – volume: 1 start-page: 213 year: 2013 ident: 1273_CR54 publication-title: Clinical Psychological Science doi: 10.1177/2167702612469015 – volume: 23 start-page: 184 year: 2014 ident: 1273_CR44 publication-title: Current Directions in Psychological Science doi: 10.1177/0963721414531598 – volume: 70 start-page: 153 year: 2017 ident: 1273_CR45 publication-title: Journal of Experimental Social Psychology doi: 10.1016/j.jesp.2017.01.006 – volume: 50 start-page: 1953 year: 2018 ident: 1273_CR4 publication-title: Behavior Research Methods doi: 10.3758/s13428-017-0963-x – volume: 1 start-page: 59 year: 2014 ident: 1273_CR40 publication-title: Journal of Experimental Political Science doi: 10.1017/xps.2014.7 – ident: 1273_CR35 doi: 10.1037/pag0000285 – volume: 5 start-page: 1 issue: 4677 year: 2014 ident: 1273_CR50 publication-title: Nature Communications doi: 10.1038/ncomms4677 – volume: 22 start-page: 1 year: 2007 ident: 1273_CR27 publication-title: Mind & Language doi: 10.1111/j.1468-0017.2006.00297.x – volume: 6 start-page: 1 year: 2016 ident: 1273_CR41 publication-title: Sage Open doi: 10.1177/2158244016636433 – volume: 6 start-page: 1 issue: 1 year: 2019 ident: 1273_CR13 publication-title: Research & Politics doi: 10.1177/2053168018822174 – volume: 77 start-page: 184 year: 2017 ident: 1273_CR58 publication-title: Computers in Human Behavior doi: 10.1016/j.chb.2017.08.038 – ident: 1273_CR48 – volume: 48 start-page: 400 year: 2016 ident: 1273_CR28 publication-title: Behavior Research Methods doi: 10.3758/s13428-015-0578-z – start-page: 19 volume-title: Virtues and vices: And other essays in moral philosophy year: 1978 ident: 1273_CR21 – start-page: 114 volume-title: Handbook of personality: Theory and research year: 2008 ident: 1273_CR36 – ident: 1273_CR62 – volume: 15 start-page: e260 year: 2013 ident: 1273_CR14 publication-title: Journal of Medical Internet Research doi: 10.2196/jmir.2903 – volume: 2 start-page: 1 issue: 4 year: 2015 ident: 1273_CR10 publication-title: Research & Politics doi: 10.1177/2053168015622072 – volume: 14 start-page: 399 year: 2011 ident: 1273_CR32 publication-title: Experimental Economics doi: 10.1007/s10683-011-9273-9 – volume: 26 start-page: 213 year: 2013 ident: 1273_CR23 publication-title: Journal of Behavioral Decision Making doi: 10.1002/bdm.1753 – volume: 41 start-page: 63 year: 2001 ident: 1273_CR38 publication-title: Journal of Advertising Research doi: 10.2501/JAR-41-5-63-71 – ident: 1273_CR20 – volume: 66 start-page: 877 year: 2015 ident: 1273_CR25 publication-title: Annual Review of Psychology doi: 10.1146/annurev-psych-010814-015321 – start-page: 219 volume-title: Online Panel Research year: 2014 ident: 1273_CR31 doi: 10.1002/9781118763520.ch10 – ident: 1273_CR47 – volume: 10 start-page: 479 year: 2015 ident: 1273_CR56 publication-title: Judgment and Decision Making doi: 10.1017/S1930297500005611 – volume: 33 start-page: 94 year: 2010 ident: 1273_CR26 publication-title: Behavioral and Brain Sciences doi: 10.1017/S0140525X10000300 – start-page: 135 volume-title: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining year: 2018 ident: 1273_CR18 doi: 10.1145/3159652.3159661 – volume: 59 start-page: 204 year: 1976 ident: 1273_CR59 publication-title: The Monist doi: 10.5840/monist197659224 – volume: 44 start-page: 196 year: 2017 ident: 1273_CR24 publication-title: Journal of Consumer Research doi: 10.1093/jcr/ucx047 – ident: 1273_CR2 doi: 10.1073/pnas.1819310116 – volume: 6 start-page: 3 year: 2011 ident: 1273_CR5 publication-title: Perspectives on Psychological Science doi: 10.1177/1745691610393980 – volume: 7 start-page: 1 issue: 2 year: 2017 ident: 1273_CR7 publication-title: SAGE Open doi: 10.1177/2158244017712774 – volume: 115 start-page: 12441 year: 2018 ident: 1273_CR12 publication-title: Proceedings of the National Academy of Sciences doi: 10.1073/pnas.1808083115 – ident: 1273_CR57 – start-page: 102 volume-title: Handbook of personality: Theory and research year: 1999 ident: 1273_CR37 – ident: 1273_CR11 – volume: 19 start-page: 25 year: 2005 ident: 1273_CR22 publication-title: Journal of Economic Perspectives doi: 10.1257/089533005775196732 – volume: 113 start-page: 858 year: 2017 ident: 1273_CR16 publication-title: Journal of Personality and Social Psychology doi: 10.1037/pspa0000105 – ident: 1273_CR49 – volume: 21 start-page: 736 year: 2017 ident: 1273_CR55 publication-title: Trends in Cognitive Sciences doi: 10.1016/j.tics.2017.06.007 – volume: 211 start-page: 453 year: 1981 ident: 1273_CR61 publication-title: Science doi: 10.1126/science.7455683 – volume: 46 start-page: 141 year: 2017 ident: 1273_CR39 publication-title: Journal of Advertising doi: 10.1080/00913367.2016.1269304 – volume: 25 start-page: 1968 year: 2017 ident: 1273_CR64 publication-title: Psychonomic Bulletin & Review doi: 10.3758/s13423-017-1348-y – start-page: 23 volume-title: Online panel research: A data quality perspective year: 2014 ident: 1273_CR6 doi: 10.1002/9781118763520.ch2 – ident: 1273_CR1 doi: 10.3886/ICPSR36390.v1 – volume: 47 start-page: 519 year: 2015 ident: 1273_CR42 publication-title: Behavior Research Methods doi: 10.3758/s13428-014-0483-x |
| SSID | ssj0038214 |
| Score | 2.6786604 |
| Snippet | Amazon Mechanical Turk (MTurk) is widely used by behavioral scientists to recruit research participants. MTurk offers advantages over traditional student... |
| SourceID | pubmedcentral proquest pubmed crossref springer |
| SourceType | Open Access Repository Aggregation Database Index Database Enrichment Source Publisher |
| StartPage | 2022 |
| SubjectTerms | Attention Behavioral Research - methods Behavioral Science and Psychology Cognitive Psychology Crowdsourcing Data Accuracy Humans Internet Mass Screening Psychology Religiosity Reproducibility of Results Social Sciences Students |
| Title | Online panels in social science research: Expanding sampling methods beyond Mechanical Turk |
| URI | https://link.springer.com/article/10.3758/s13428-019-01273-7 https://www.ncbi.nlm.nih.gov/pubmed/31512174 https://www.proquest.com/docview/2306531289 https://www.proquest.com/docview/2289577253 https://pubmed.ncbi.nlm.nih.gov/PMC6797699 |
| Volume | 51 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3dS8MwED_Uvfgiflu_iOCbFpvmq_VtjM2hzCcHig-liSmKUsVtoP-9l6arTlHwpRRyIc1dLnfX5H4HcGgjjUFGokOTax5yYXiYsCgKTVFwlqBCyQpne3Ap-0N-fi2ua5gclwvz5fyeoSt7MqKMOwxll2pD0dSGah5agjLlyjR0ZGe667IkptwnxfzSb9bw_PAmf16K_HYyWhmc3jIs1Z4iaXvRrsCcLVdhsdmw3tfg1uOEElRotHDkoST-DzipzRqpkXzuT0n37cXnr5BR7u6Q44uvHT0iusphIQPrcoCdyMjV5PVxHYa97lWnH9bFEkIjIzUOJdd3Ii0io5U0xsZ33EphnD_GEplLLXOlqSiYFlQxmVpqlZQ5iwsdG5nyiG3AQvlc2i0gOXahmlON3g3XTGmjUpUz7krBYNBZBECnnMxMjSTuClo8ZRhROO5nnvsZcj-ruJ-pAI6aPi8eR-NP6t2pgLJap0ZZVeSeoT1NAzhomlEb3BEHMvp5gjTYKjBgECyATS_PZjjmnBucUQBqRtINgUPanm0pH-4rxG2JDJApjns8XROfn_X7LLb_R74Di7Fbr9VNwV1YGL9O7B56PGO9D6322c1Fd79a8vgcxu0Pmmr60A |
| linkProvider | Springer Nature |
| linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LT9wwEB619AAXVF4lLRQjcYOocezYSW8IgbbActqVkDhYsXG0CBRWm12p_fcd20mqBVGJWyTbcjIPz0w88w3AkU00Bhm5jk2pecwzw-OcJUlsqoqzHBVKeJzt4Y0YjPnlbXbbFoU1XbZ7dyXpT2oXV6JT-6OhjDs0ZVd0Q9HoxvIjfEJnIHeyPE5Pu_OX5SnloTzmjXXLJuiVX_k6PfLFHak3PRefYb31GclpYPIGfLD1Jqz1R9efLbgLiKEEVRttHXmoSfgXTloDR1pMn8lPcv57GipZSFO6bHJ8CF2kG6J9NQsZWlcN7JhHRovZ4zaML85HZ4O4bZsQG5HIeSy4vs-KKjFaCmNses-tyIzzzFguSqFFKTXNKqYzKpkoLLVSiJKllU6NKHjCdmClfq7tLpASl1DNqUY_h2smtZGFLBl3TWEw_KwioB0llWkxxV1riyeFsYWjvgrUV0h95amvZATH_ZppQNT47-y9jkGq1a5G-Xb3DC1rEcFhP4x64S47kNDPC5yDoxmGDhmL4EvgZ78dc24OflEEconT_QSHub08Uj9MPPa2QAKIAvc96WTi32u9_RVf3zf9AFYHo-G1uv51c_UN1lInuz5_cA9W5rOF3Uc_aK6_e7H_C2xC_2o |
| linkToPdf | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3dT9wwDLe2m4R4QWOMUTi2IPE2KpomTdq9TTdO7APEA0hIPERNLhVoUzlxdxL899hN2-2GmLS3SnGU1o5ju7F_Btj3icUgI7exK62MZeZknIskiV1VSZGjQqkGZ_vkVB1fyG-X2eUfVfxNtnt3JRlqGgilqZ4fTicVqbhAB_dwxoUkZGUqwOFogGP9El5hpMIp_BqpUXcWizzlMpTKPDNv2Rw98TGfpkr-dV_amKHxa1hr_Uf2OQh8HV74-g2s9sfYwwZcBfRQhmqOdo_d1Cz8F2etsWMtvs_1J3Z0Pw1VLWxWUmY5PoSO0jNmm8oWduKpMpgEyc4Xdz_fwsX46Hx0HLctFGKnEj2PlbSTrKgSZ7VyzqcT6VXmyEsTuSqVVaW2PKuEzbgWqvDca6VKkVY2daqQidiEQX1b-y1gJU7hVnKLPo-0QlunC10KSQ1iMBStIuAdJ41r8cWpzcUvg3EGcd8E7hvkvmm4b3QEH_s504Cu8U_qYScg02razFAIhecIxo0R7PXDqCN08YGMvl0gDY5mGEZkIoJ3QZ79coJcHvyiCPSSpHsCwt9eHqlvrhscboUMUAWue9Dtid-v9fxXbP8f-QdYOfsyNj--nn7fgdWUtm6TSjiEwfxu4XfRJZrb982ufwTMUgOf |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Online+panels+in+social+science+research%3A+Expanding+sampling+methods+beyond+Mechanical+Turk&rft.jtitle=Behavior+research+methods&rft.au=Chandler%2C+Jesse&rft.au=Rosenzweig%2C+Cheskie&rft.au=Moss%2C+Aaron+J&rft.au=Robinson%2C+Jonathan&rft.date=2019-10-01&rft.issn=1554-3528&rft.eissn=1554-3528&rft.volume=51&rft.issue=5&rft.spage=2022&rft_id=info:doi/10.3758%2Fs13428-019-01273-7&rft.externalDBID=NO_FULL_TEXT |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1554-3528&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1554-3528&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1554-3528&client=summon |