Examination of Common Exams Held by Measurement and Assessment Centers: Many Facet Rasch Analysis

• Published in: International journal of assessment tools in education Vol. 8; no. 3; pp. 649 - 666
• Main Authors: KAYA UYANIK, Gülden, DEMİRTAŞ TOLAMAN, Tuğba, GÜR ERDOĞAN, Duygu
• Format: Journal Article
• Language: English
• Published: International Journal of Assessment Tools in Education 01.01.2021
• Subjects: Difficulty Level; Evaluation Methods; Foreign Countries; Grade 6; Item Response Theory; Measurement Techniques; Standardized Tests; Test Items; Test Theory; Thinking Skills; Turkey
• ISSN: 2148-7456; 2148-7456
• DOI: 10.21449/ijate.730977

This paper aims to examine and assess the questions included in the “Turkish Common Exam” for sixth graders held in the first semester of 2018 which is one of the common exams carried out by The Measurement and Evaluation Centers, in terms of question structure, quality and taxonomic value. To this end, the test questions were examined by three specialists with expertise in different fields in terms of structure, content, and taxonomic values. The test questions were then rated by raters with expertise in different fields according to the criteria set by the researchers. Hence, the study employed the descriptive survey model. The data obtained from the assessment of the questions were analyzed using the Many Facet Rasch Model (MFRM). According to the findings, of the 20 questions included in the exam, 5 (five) are in the category of “Remembering”, 12 (twelve) in the category of “Understanding”, 2 (two) in the category of “Analyzing” and 1 (one) in the category of “Evaluating.” Accordingly, the number of questions that measure higher-order thinking skills was lower than the number of lower-level questions. In addition, the study contained three facets: raters, tasks (items), and criteria. There were no differences among the raters (a Turkish Education Specialist, a Program Development Specialist, and a Testing and Assessment Specialist) in terms of severity and leniency: all the raters were in agreement. Finally, in this study, the questions met the criteria measuring the structural features, while they failed to meet the criteria measuring the quality and clarity. This paper aims to examine and assess the questions included in the “Turkish Common Exam” for sixth graders held in the first semester of 2018 which is one of the common exams carried out by The Measurement and Evaluation Centers, in terms of question structure, quality and taxonomic value. To this end, the test questions were examined by three specialists with expertise in different fields in terms of structure, content, and taxonomic values. The test questions were then rated by raters with expertise in different fields according to the criteria set by the researchers. Hence, the study employed the descriptive survey model. The data obtained from the assessment of the questions were analyzed using the Many Facet Rasch Model (MFRM). According to the findings, of the 20 questions included in the exam, 5 (five) are in the category of “Remembering”, 12 (twelve) in the category of “Understanding”, 2 (two) in the category of “Analyzing” and 1 (one) in the category of “Evaluating.” Accordingly, the number of questions that measure higher-order thinking skills was lower than the number of lower-level questions. In addition, the study contained three facets: raters, tasks (items), and criteria. There were no differences among the raters (a Turkish Education Specialist, a Program Development Specialist, and a Testing and Assessment Specialist) in terms of severity and leniency: all the raters were in agreement. Finally, in this study, the questions met the criteria measuring the structural features, while they failed to meet the criteria measuring the quality and clarity.