A meta-analysis of test format effects on reading and listening test performance: Focus on multiple-choice and open-ended formats

A meta-analysis was conducted on the effects of multiple-choice and open-ended formats on L1 reading, L2 reading, and L2 listening test performance. Fifty-six data sources located in an extensive search of the literature were the basis for the estimates of the mean effect sizes of test format effect...

Full description

Saved in:
Bibliographic Details
Published inLanguage testing Vol. 26; no. 2; pp. 219 - 244
Main Authors In'nami, Yo, Koizumi, Rie
Format Journal Article
LanguageEnglish
Published London, England SAGE Publications 01.04.2009
Sage Publications Ltd
Subjects
Dix, Dorothea Lynde (1802-87)
Educational evaluation
Effect Size
Handbooks
Language
Language Proficiency
Language Tests
Linguistics
Listening
Listening Comprehension
Listening Comprehension Tests
Literature reviews
Mental disorders
Meta Analysis
Multiple Choice Tests
Program Effectiveness
Reading comprehension
Reading Tests
Second Language Learning
Second language reading
Studies
Systematic review
Test Format
Test Validity and Reliability
New York
United Kingdom > UK
United States > US
Japan
test method
listening test
multiple-choice
meta-analysis
reading test
test format
open-ended
Online AccessGet full text

Cover

More Information
Summary:A meta-analysis was conducted on the effects of multiple-choice and open-ended formats on L1 reading, L2 reading, and L2 listening test performance. Fifty-six data sources located in an extensive search of the literature were the basis for the estimates of the mean effect sizes of test format effects. The results using the mixed effects model of meta-analysis indicate that multiple-choice formats are easier than open-ended formats in L1 reading and L2 listening, with the degree of format effect ranging from small to large in L1 reading and medium to large in L2 listening. Overall, format effects in L2 reading are not found, although multiple-choice formats are found to be easier than open-ended formats when any one of the following four conditions is met: the studies involve between-subjects designs, random assignment, stem-equivalent items, or learners with a high L2 proficiency level. Format effects favoring multiple-choice formats across the three domains are consistently observed when studies employ between-subjects designs, random assignment, or stem-equivalent items.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:0265-5322
1477-0946
DOI:10.1177/0265532208101006