A Randomized Experiment to Compare Conventional, Computerized, and Computerized Adaptive Administration of Ordinal Polytomous Attitude Items
A total of 520 high school students were randomly assigned to a paper-and-pencil test (PPT), a computerized standard test (CST), or a computerized adaptive test (CAT) version of the Dutch School Attitude Questionnaire (SAQ), consisting of ordinal polytomous items. The CST administered items in the s...
Saved in:
| Published in | Applied psychological measurement Vol. 29; no. 3; pp. 159 - 183 |
|---|---|
| Main Authors | , , |
| Format | Journal Article |
| Language | English |
| Published |
Thousand Oaks, CA
Sage Publications
01.05.2005
|
| Subjects | |
| Online Access | Get full text |
Cover
Loading…
| Abstract | A total of 520 high school students were randomly assigned to a paper-and-pencil test (PPT), a computerized standard test (CST), or a computerized adaptive test (CAT) version of the Dutch School Attitude Questionnaire (SAQ), consisting of ordinal polytomous items. The CST administered items in the same order as the PPT. The CAT administered all items of three SAQ subscales in adaptive order using Samejima’s graded response model, so that six different stopping rule settings could be applied afterwards. School marks were used as external criteria. Results showed significant but small multivariate administration mode effects on conventional raw scores and small to medium effects on maximum likelihood latent trait estimates. When the precision of CAT latent trait estimates decreased, correlations with grade point average in general decreased. However, the magnitude of the decrease was not very large as compared to the PPT, the CST, and the CAT without the stopping rule. |
|---|---|
| AbstractList | A total of 520 high school students were randomly assigned to a paper-and-pencil test (PPT), a computerized standard test (CST), or a computerized adaptive test (CAT) version of the Dutch School Attitude Questionnaire (SAQ), consisting of ordinal polytomous items. The CST administered items in the same order as the PPT. The CAT administered all items of three SAQ subscales in adaptive order using Samejima's graded response model, so that six different stopping rule settings could be applied afterwards. School marks were used as external criteria. Results showed significant but small multivariate administration mode effects on conventional raw scores and small to medium effects on maximum likelihood latent trait estimates. When the precision of CAT latent trait estimates decreased, correlations with grade point average in general decreased. However, the magnitude of the decrease was not very large as compared to the PPT, the CST, and the CAT without the stopping rule. |
| Audience | High Schools |
| Author | Hol, A. Michiel Mellenbergh, Gideon J. Vorst, Harrie C. M. |
| Author_xml | – sequence: 1 givenname: A. Michiel surname: Hol fullname: Hol, A. Michiel email: a.m.hol@uva.nl organization: University of Amsterdam – sequence: 2 givenname: Harrie C. M. surname: Vorst fullname: Vorst, Harrie C. M. organization: University of Amsterdam – sequence: 3 givenname: Gideon J. surname: Mellenbergh fullname: Mellenbergh, Gideon J. organization: University of Amsterdam |
| BackLink | http://eric.ed.gov/ERICWebPortal/detail?accno=EJ689712$$DView record in ERIC |
| BookMark | eNp1UF1LwzAUDTLBbfrugw_5Aave9CNtH8uYOhlMRJ_LXZNKxtqUJB3O3-CPNnUiKPh0L-cLzpmQUatbScglg2vG0vQGWMx5yDjEYcpCnp2QMUuSMIjiPB2R8UAHA39GJtZuASDieTImHwV9wlboRr1LQRdvnTSqka2jTtO5bjo00t927yGlW9zNvtDeeZk3zKj3_kJoIbBzai_906hWWWdwcFJd07URykfQR707ON3o3tLCOeV6IenSycaek9Mad1ZefN8pebldPM_vg9X6bjkvVkEVQegC5DHycJOEGUDNeCUFApdRFKLM8ziCKqtzrARu_MewAhlhDSCqDca-NeTRlMAxtzLaWiPrsvOt0RxKBuWwZvl3TW-5Olp8zepHvnjgWe4Fng6OtMVXWW51b3xR-3_cJ9vgguE |
| CitedBy_id | crossref_primary_10_1177_0146621606297314 crossref_primary_10_1177_10731911211047892 crossref_primary_10_1080_15305058_2011_561459 crossref_primary_10_1016_j_paid_2010_02_023 crossref_primary_10_1016_j_psychres_2010_12_001 |
| Cites_doi | 10.1111/j.2044-8317.1990.tb00925.x 10.1177/014662168901300202 10.1177/014662169501900110 10.1007/BF02296272 10.1007/978-1-4757-2691-6_5 10.1007/978-1-4899-2927-3 10.2307/3172903 10.1037/1040-3590.11.1.58 10.1177/014662169501900103 10.1037/0022-3514.57.6.1051 10.1007/BF02291262 10.1177/0013164403063003005 10.1177/01466219922031310 10.2307/3172588 10.1080/07481756.2002.12069050 10.1093/biomet/75.4.800 10.1177/107319110000700404 10.1037/0033-2909.114.3.449 10.4324/9781410602527 10.1177/014662167800200413 10.1037/0033-2909.97.1.148 |
| ContentType | Journal Article |
| DBID | 7SW BJH BNH BNI BNJ BNO ERI PET REK WWN AAYXX CITATION |
| DOI | 10.1177/0146621604271268 |
| DatabaseName | ERIC ERIC (Ovid) ERIC ERIC ERIC (Legacy Platform) ERIC( SilverPlatter ) ERIC ERIC PlusText (Legacy Platform) Education Resources Information Center (ERIC) ERIC CrossRef |
| DatabaseTitle | ERIC CrossRef |
| DatabaseTitleList | ERIC CrossRef |
| Database_xml | – sequence: 1 dbid: ERI name: ERIC url: https://eric.ed.gov/ sourceTypes: Index Database |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Psychology |
| EISSN | 1552-3497 |
| ERIC | EJ689712 |
| EndPage | 183 |
| ExternalDocumentID | 10_1177_0146621604271268 EJ689712 10.1177_0146621604271268 |
| GroupedDBID | --Z -MK -TM -~X .2G .2J .2L .2N 01A 09Z 0R~ 1~K 23M 31S 31U 31V 31W 31X 31Y 31Z 4.4 56W 5GY 5VS AAAHA AABMB AABOD AACKU AACMV AACTG AADIR AADTT AADUE AAEWN AAGGD AAJIQ AAJOX AAJPV AAKTJ AAMFR AAMGE AANSI AAPEO AAQDB AAQXH AARDL AARIX AATAA AATBZ AAWLO AAYTG ABAWP ABCCA ABDLQ ABDWY ABEIX ABFWQ ABFXH ABHKI ABHQH ABIVO ABJNI ABKRH ABLUO ABPGX ABPNF ABQKF ABQPY ABQXT ABRHV ABTAH ABTDE ABVFX ABYTW ACABN ACAEP ACARO ACDSZ ACDXX ACFEJ ACFMA ACFUR ACFZE ACGBL ACGFS ACGZU ACHQT ACLHI ACLZU ACNCT ACOFE ACOXC ACROE ACSBE ACSIQ ACTQU ACUAV ACUFS ACUIR ACXKE ACXMB ADBBV ADEIA ADNWM ADPEE ADRRZ ADTBJ ADTOS ADUKL AECGH AECVZ AEDTQ AEOBU AEONT AEPTA AEQLS AERKM AESMA AESZF AEUHG AEUIJ AEVPJ AEWDL AEWHI AEXNY AFEET AFFNX AFKBI AFKRG AFMOU AFQAA AFUIA AGDVU AGKLV AGNHF AGNWV AGWFA AHHFK AHOJL AHWHD AIOMO AJEFB AJMMQ AJUZI AJXAJ ALJHS ALMA_UNASSIGNED_HOLDINGS AMCVQ ANDLU AOIJS ARTOV ASPBG AUTPY AUVAJ AVWKF AYAKG AYPQM AZFZN B8O B8R B8S B8T B8Z B93 B94 BBRGL BDDNI BDZRT BKIIM BMVBW BPACV BSEHC BYIEH CAG CBRKF CCGJY CEADM CFDXU COF CORYS CQQTX CS3 DC- DC. DD- DD0 DD~ DE- DF0 DG~ DO- DOPDO DV7 DV8 D~Y EBS EJD F5P FEDTE FHBDP FRS GROUPED_SAGE_PREMIER_JOURNAL_COLLECTION HF~ HVGLF HYE HZ~ J8X JCYGO K.F K.J LPU M4V MS~ N9A O9- OK1 P.B P2P PQQKQ Q1R Q7K Q7L Q7O Q7P Q7X Q82 Q83 ROL RPM S01 SBI SCNPE SDB SFB SFC SFK SFT SFX SFY SGA SGP SGR SGU SGV SGX SGZ SHB SHF SHG SHM SNB SPJ SPP SPV SQCSI SSDHQ STM TN5 UAP WH7 YYP ZONMY ZPLXX ZPPRI ZRKOI ZXP ZY4 ~32 ~A~ 7SW AEDXQ ALKWR BJH BNH BNI BNJ BNO ERI H13 PET REK WWN AAYXX ACJER CITATION |
| ID | FETCH-LOGICAL-c302t-a64a62b52800f16ceda06e332ae99430c8f9acdab0c81ac0e3af00dcba4036093 |
| ISSN | 0146-6216 |
| IngestDate | Wed Sep 25 14:09:42 EDT 2024 Fri Sep 06 12:34:57 EDT 2024 Tue Jul 16 20:42:58 EDT 2024 |
| IsDoiOpenAccess | false |
| IsOpenAccess | false |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 3 |
| Keywords | statistical validity test administration computer assisted testing Likert scales item response theory attitude measurement adaptive testing |
| Language | English |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-c302t-a64a62b52800f16ceda06e332ae99430c8f9acdab0c81ac0e3af00dcba4036093 |
| PageCount | 25 |
| ParticipantIDs | crossref_primary_10_1177_0146621604271268 eric_primary_EJ689712 sage_journals_10_1177_0146621604271268 |
| PublicationCentury | 2000 |
| PublicationDate | 20050500 2005-05-00 |
| PublicationDateYYYYMMDD | 2005-05-01 |
| PublicationDate_xml | – month: 05 year: 2005 text: 20050500 |
| PublicationDecade | 2000 |
| PublicationPlace | Thousand Oaks, CA |
| PublicationPlace_xml | – name: Thousand Oaks, CA |
| PublicationTitle | Applied psychological measurement |
| PublicationYear | 2005 |
| Publisher | Sage Publications |
| Publisher_xml | – name: Sage Publications |
| References | McCoach, D. B., Siegle, D. 2003; 63 Meijer, R. R., Nering, M. L. 1999; 23 Dodd, B. G., Koch, W. R., De Ayala, R. J. 1989; 13 Baek, S. G. 1995; 55 Bock, R. D., Lieberman, M. 1970; 35 Reise, S. P., Henson, J. M. 2000; 7 McCoach, D. B. 2002; 35 Mellenbergh, G. J. 1995; 19 Hochberg, Y. 1988; 75 Hol, A. M., Vorst, H. C. M., Mellenbergh, G. J. 2001; 56 Zwick, R. 1985; 97 Balasubramanian, S. K., Kamakura, W. A. 1989; 26 Tutz, G. 1990; 43 Masters, G. N. 1982; 47 Dodd, B. G., De Ayala, R. J., Koch, W. R. 1995; 19 Singh, J., Howell, R. D., Rhoads, G. K. 1990; 27 Waller, N. G., Reise, S. P. 1989; 57 Mead, A. D., Drasgow, F. 1993; 114 Koch, W. R., Dodd, B. G., Fitzpatrick, S. J. 1990; 23 Andrich, D. 1978; 2 Finger, M. S., Ones, D. S. 1999; 11 atypb8 Cohen, J. (atypb5) 1988 atypb19 Vorst, H. C. M. (atypb29) 2000 Baek, S. G. (atypb2) 1995; 55 atypb28 atypb22 atypb24 atypb25 atypb20 atypb21 Samejima, F. (atypb23) 1969 Thissen, D. (atypb27) 1991 Waller, N. G. (atypb30) 1999 Embretson, S. E. (atypb9) 2000 Gibbons, J. D. (atypb12) 1985 atypb16 atypb17 atypb18 atypb11 atypb33 Koch, W. R. (atypb15) 1990; 23 atypb13 Hol, A. M. (atypb14) 2001; 56 atypb1 Stevens, J. (atypb26) 1996 atypb3 atypb31 atypb10 atypb32 atypb4 atypb7 atypb6 |
| References_xml | – volume: 13 start-page: 129 year: 1989 end-page: 143 article-title: Operational characteristics of adaptive testing procedures using the graded response model publication-title: Applied Psychological Measurement contributor: fullname: De Ayala, R. J. – volume: 75 start-page: 800 year: 1988 end-page: 802 article-title: A sharper Bonferroni procedure for multiple tests of significance publication-title: Biometrika contributor: fullname: Hochberg, Y. – volume: 97 start-page: 148 year: 1985 end-page: 152 article-title: Nonparametric one-way multivariate analysis of variance: A computational approach based on the Pillai-Bartlett trace publication-title: Psychological Bulletin contributor: fullname: Zwick, R. – volume: 19 start-page: 5 year: 1995 end-page: 22 article-title: Computerized adaptive testing with polytomous items publication-title: Applied Psychological Measurement contributor: fullname: Koch, W. R. – volume: 26 start-page: 311 year: 1989 end-page: 326 article-title: Measuring consumer attitudes toward the marketplace with tailored interviews publication-title: Journal of Marketing Research contributor: fullname: Kamakura, W. A. – volume: 56 start-page: 119 year: 2001 end-page: 133 article-title: Toepassing van een computergestuurde adaptieve testprocedure op persoonlijkheidsdata [Application of a computerized adaptive test procedure on personality data] publication-title: Nederlands Tijdschrift Voor de Psychologie contributor: fullname: Mellenbergh, G. J. – volume: 35 start-page: 179 year: 1970 end-page: 197 article-title: Fitting a response model for n dichotomously scored items publication-title: Psychometrika contributor: fullname: Lieberman, M. – volume: 23 start-page: 20 year: 1990 end-page: 30 article-title: Computerized adaptive measurements of attitudes publication-title: Measurement and Evaluation in Counseling and Development contributor: fullname: Fitzpatrick, S. J. – volume: 19 start-page: 91 year: 1995 end-page: 100 article-title: Conceptual notes on models for discrete polytomous item responses publication-title: Applied Psychological Measurement contributor: fullname: Mellenbergh, G. J. – volume: 55 start-page: 1922 issue: 7-A year: 1995 article-title: Computerized adaptive attitude testing using the partial credit model publication-title: Dissertation Abstracts International—A contributor: fullname: Baek, S. G. – volume: 35 start-page: 66 year: 2002 end-page: 77 article-title: A validation study of the school attitude assessment survey publication-title: Measurement and Evaluation in Counseling and Development contributor: fullname: McCoach, D. B. – volume: 23 start-page: 187 year: 1999 end-page: 194 article-title: Computerized adaptive testing: Overview and introduction publication-title: Applied Psychological Measurement contributor: fullname: Nering, M. L. – volume: 47 start-page: 149 year: 1982 end-page: 174 article-title: A Rasch model for partial credit scoring publication-title: Psychometrika contributor: fullname: Masters, G. N. – volume: 43 start-page: 39 year: 1990 end-page: 55 article-title: Sequential item response models with an ordered response publication-title: British Journal of Mathematical and Statistical Psychology contributor: fullname: Tutz, G. – volume: 63 start-page: 414 year: 2003 end-page: 429 article-title: The School Attitude Assessment Survey-Revised: A new instrument to identify academically able students who underachieve publication-title: Educational and Psychological Measurement contributor: fullname: Siegle, D. – volume: 114 start-page: 449 year: 1993 end-page: 458 article-title: Equivalence of computerized and paper-and-pencil cognitive ability tests: A meta-analysis publication-title: Psychological Bulletin contributor: fullname: Drasgow, F. – volume: 7 start-page: 347 year: 2000 end-page: 364 article-title: Computerization and adaptive administration of the NEO PI-R publication-title: Assessment contributor: fullname: Henson, J. M. – volume: 2 start-page: 581 year: 1978 end-page: 594 article-title: Application of a psychometric rating model to ordered categories which are scored with successive integers publication-title: Applied Psychological Measurement contributor: fullname: Andrich, D. – volume: 11 start-page: 58 year: 1999 end-page: 66 article-title: Psychometric equivalence of the computer and booklet forms of the MMPI: A meta-analysis publication-title: Psychological Assessment contributor: fullname: Ones, D. S. – volume: 27 start-page: 304 year: 1990 end-page: 321 article-title: Adaptive designs for Likert-type data: An approach for implementing marketing research publication-title: Journal of Marketing Research contributor: fullname: Rhoads, G. K. – volume: 57 start-page: 1051 year: 1989 end-page: 1058 article-title: Computerized adaptive personality assessment: An illustration with the Absorption Scale publication-title: Journal of Personality and Social Psychology contributor: fullname: Reise, S. P. – ident: atypb28 doi: 10.1111/j.2044-8317.1990.tb00925.x – ident: atypb7 doi: 10.1177/014662168901300202 – volume: 23 start-page: 20 year: 1990 ident: atypb15 publication-title: Measurement and Evaluation in Counseling and Development contributor: fullname: Koch, W. R. – ident: atypb21 doi: 10.1177/014662169501900110 – volume-title: Applied multivariate statistics for the social sciences year: 1996 ident: atypb26 contributor: fullname: Stevens, J. – ident: atypb16 doi: 10.1007/BF02296272 – volume-title: Statistical power analysis for the behavioral sciences year: 1988 ident: atypb5 contributor: fullname: Cohen, J. – ident: atypb24 doi: 10.1007/978-1-4757-2691-6_5 – ident: atypb10 doi: 10.1007/978-1-4899-2927-3 – ident: atypb3 doi: 10.2307/3172903 – ident: atypb11 doi: 10.1037/1040-3590.11.1.58 – volume-title: Multilog 6.0 user’s guide year: 1991 ident: atypb27 contributor: fullname: Thissen, D. – ident: atypb6 doi: 10.1177/014662169501900103 – volume-title: Schoolvragenlijst [School Attitude Questionnaire] year: 2000 ident: atypb29 contributor: fullname: Vorst, H. C. M. – volume-title: Psychometrika Monograph year: 1969 ident: atypb23 contributor: fullname: Samejima, F. – ident: atypb32 – ident: atypb31 doi: 10.1037/0022-3514.57.6.1051 – volume-title: Item response theory for psychologists year: 2000 ident: atypb9 contributor: fullname: Embretson, S. E. – ident: atypb4 doi: 10.1007/BF02291262 – volume-title: Nonparametric statistical inference year: 1985 ident: atypb12 contributor: fullname: Gibbons, J. D. – ident: atypb18 doi: 10.1177/0013164403063003005 – ident: atypb20 doi: 10.1177/01466219922031310 – ident: atypb25 doi: 10.2307/3172588 – start-page: 185 volume-title: The new rules of measurement: What every psychologist and educator should know year: 1999 ident: atypb30 contributor: fullname: Waller, N. G. – volume: 55 start-page: 1922 issue: 7 year: 1995 ident: atypb2 publication-title: Dissertation Abstracts International—A contributor: fullname: Baek, S. G. – ident: atypb17 doi: 10.1080/07481756.2002.12069050 – ident: atypb13 doi: 10.1093/biomet/75.4.800 – ident: atypb22 doi: 10.1177/107319110000700404 – ident: atypb19 doi: 10.1037/0033-2909.114.3.449 – ident: atypb8 doi: 10.4324/9781410602527 – ident: atypb1 doi: 10.1177/014662167800200413 – volume: 56 start-page: 119 year: 2001 ident: atypb14 publication-title: Nederlands Tijdschrift Voor de Psychologie contributor: fullname: Hol, A. M. – ident: atypb33 doi: 10.1037/0033-2909.97.1.148 |
| SSID | ssj0003695 |
| Score | 1.7483048 |
| Snippet | A total of 520 high school students were randomly assigned to a paper-and-pencil test (PPT), a computerized standard test (CST), or a computerized adaptive... |
| SourceID | crossref eric sage |
| SourceType | Aggregation Database Index Database Publisher |
| StartPage | 159 |
| SubjectTerms | Adaptive Testing Attitude Measures Computer Assisted Testing High School Students Item Response Theory Psychological Studies School Attitudes Standardized Tests |
| Title | A Randomized Experiment to Compare Conventional, Computerized, and Computerized Adaptive Administration of Ordinal Polytomous Attitude Items |
| URI | https://journals.sagepub.com/doi/full/10.1177/0146621604271268 http://eric.ed.gov/ERICWebPortal/detail?accno=EJ689712 |
| Volume | 29 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV1La9tAEF5McsmlNG1K3aZhDyVQHLmrh9feozBujcFtWpKSnsxqH2BoLOMoh-Q3FPqXO_vQsw2kvQixSIPY-TQ7M_vtDEJvGTdF0TkNGCE6SLSIzT83DrSMqeTxROrM5DuWn-j8Mllcja56vV8N1tJtkQ3F_V_PlfyPVmEM9GpOyf6DZiuhMAD3oF-4gobh-igdp4OvfCPz6_U9uI2zqla_8SenjltuTvRVrHKbFPVdHMwrJXOzOTZIJd9aOlG7rK5xKj8DmIzzep7_uCtykzMYpIWhGkhlU_43TU-3dG-3LQt7XackK0C5Hl_p0FL416qifHzLd-5AypzvIKAfTId16nZp9hssN82mhT6upYJvXLRyGKOaMTjs0pO6aUqw5AGN3HnM0mL7HMm6GdBb8xu66uJ_Lgt2Y9rIMqJMe5Ewct18OsW2Zws6YWPTtXo_AssFJnM__X7-ZVkt7jFlnhXrPqve-X7fFW-rHThxLafH8-obxEHry1w8RU98EIJTh6hD1FObZ-igWgvvnqOfKa6hhWto4SLHHlq4Ca0z3ATRGYZ3WyO4hBVuwwrnGntY4RpWuIQVtrA6QpcfZhfTeeAbdwQiJlERcJpwGmWjCKIRHVKhJCdUxXHEFTPl_sVEMy4kz-Au5IKomGtCpMh4AhNMWPwC7W3yjXqJcDROsoxmEHiDHynJhBGmNYTRIyYE05L30btyZldbV59lFZYl7DsK6aMjM_XVc6V2-ujU6GLl_-ybBwW8ekDAa3RQo_oY7RW7W_UGfNUiO_H4ObF5md8QYpZX |
| link.rule.ids | 220,315,783,787,27936,27937 |
| linkProvider | SAGE Publications |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+Randomized+Experiment+to+Compare+Conventional%2C+Computerized%2C+and+Computerized+Adaptive+Administration+of+Ordinal+Polytomous+Attitude+Items&rft.jtitle=Applied+psychological+measurement&rft.au=Hol%2C+A.+Michiel&rft.au=Vorst%2C+Harrie+C.+M&rft.au=Mellenbergh%2C+Gideon+J&rft.date=2005-05-01&rft.pub=Sage+Publications&rft.issn=0146-6216&rft.volume=29&rft.issue=3&rft.spage=159&rft_id=info:doi/10.1177%2F0146621604271268&rft.externalDocID=EJ689712 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0146-6216&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0146-6216&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0146-6216&client=summon |