The dependence on mathematical theory in TIMSS, PISA and TIMSS Advanced test items and its relation to student achievement

• Published in: Large-scale assessments in education Vol. 6; no. 1; pp. 1 - 17
• Main Authors: Hole, Arne, Grønmo, Liv Sissel, Onstad, Torgeir
• Format: Journal Article
• Language: English
• Published: New York Springer US 04.05.2018; Springer; Springer Nature B.V; SpringerOpen
• Subjects: Achievement Tests; Assessment; Differences; Difficulty Level; Education; Educational Policy and Politics; Educational Research; Elementary Secondary Education; Foreign Countries; Grade 8; Humanities; International Assessment; Item categorization; Item Response Theory; Large-scale assessments; Law; Mathematical competencies; Mathematical theory; Mathematics; Mathematics Achievement; Mathematics Education; Mathematics Instruction; Mathematics Tests; Physics education; Regional Characteristics; Science Achievement; Science Tests; Secondary School Students; Statistics for Social Sciences; Test Items; Testing and Evaluation; Theories; Mathematical competencies; PISA; Mathematics assessment; Mathematical theory; TIMSS; Mathematics education; Physics education; Large-scale assessments; Item categorization; TIMSS Advanced
• ISSN: 2196-0739; 2196-0739
• DOI: 10.1186/s40536-018-0055-0

Background This paper discusses a framework for analyzing the dependence on mathematical theory in test items, that is, a framework for discussing to what extent knowledge of mathematical theory is helpful for the student in solving the item. The framework can be applied to any test in which some knowledge of mathematical theory may be useful, both within mathematics itself and in other subjects. The relevance of the framework is related to the important distinction between language and content in mathematical theory. Method We used groups of scorers categorizing test items from TIMSS grade 8, PISA, and TIMSS Advanced. Differences in results across countries and groups of countries are analyzed. Results Our results indicate, among other things, that the dependence on mathematical theory in the set of TIMSS Advanced 2015 physics test items is greater than in the set of PISA 2012 mathematics test items. Concerning relations to item difficulty, we find interesting differences in average p -values between the sets of item groups defined by our framework concerning participating countries and geographically defined groups of countries. Conclusions The results indicate deep differences in mathematics teaching traditions and curricula in different regions of the world. Documenting such differences may help different educational systems learn from each other, and as such is relevant for all forms of educational research.