PISA Country Rankings Valid? Results for Canada and Finland

This article examines whether the way that PISA models item outcomes in mathematics affects the validity of its country rankings. As an alternative to PISA methodology a two-parameter model is applied to PISA mathematics item data from Canada and Finland for the year 2012. In the estimation procedur...

Full description

Saved in:
Bibliographic Details
Published inScandinavian journal of educational research Vol. 63; no. 5; pp. 670 - 678
Main Author McIntosh, James
Format Journal Article
LanguageEnglish
Published Abingdon Routledge 29.07.2019
Taylor & Francis Ltd
Subjects
Achievement Tests
Canada
Comparative analysis
Comparative Education
Cross Cultural Studies
Difficulty Level
Educational Policy
Finland
Foreign Countries
International Assessment
Item Analysis
item response models
Item Response Theory
Mathematics
Mathematics Achievement
PISA
Respondents
Scores
Secondary School Students
Test Items
test score
Canada
Finland
Online AccessGet full text

Cover

More Information
Summary:This article examines whether the way that PISA models item outcomes in mathematics affects the validity of its country rankings. As an alternative to PISA methodology a two-parameter model is applied to PISA mathematics item data from Canada and Finland for the year 2012. In the estimation procedure item difficulty and dispersion parameters are allowed to differ across the two countries and samples are restricted to respondents who actually answered items in a mathematics cluster. Different normalizations for identifying the distribution parameters are also considered. The choice of normalization is shown to be crucial in guaranteeing certain invariance properties required by item response models. The ability scores obtained from the methods employed here are significantly higher for Finland, in sharp contrast to PISA results, which gave both countries very similar ranks in mathematics.
ISSN:0031-3831
1470-1170
DOI:10.1080/00313831.2017.1420687