An eight-category partial credit model as very appropriate for four-tier diagnostic test scoring in physics learning

• Published in: AIP conference proceedings Vol. 2622; no. 1
• Main Authors: Istiyono, Edi, Fenditasari, Kharisma, Ayub, Made Rai Shanti, Saepuzaman, Duden, Dwandaru, Wipsar Sunu Brams
• Format: Journal Article Conference Proceeding
• Language: English
• Published: Melville American Institute of Physics 29.04.2024
• Subjects: Categories; Diagnostic systems; Diagnostic tests; Estimates; Learning; Physics; Questions; Scoring models; Students
• ISSN: 0094-243X; 1551-7616
• DOI: 10.1063/5.0133862

There have been many two or three tier diagnostic tests with classical test theory, in this study a fourtier diagnostic test was developed which was analyzed using item response theory. This study aims (i) to find the instrument construct of the Four Tier Diagnostic Test (FTDT), (ii) to design the FTDT scoring model, (iii) Identify the characteristics of the instrument. This FTDT development model uses a 4D model. The instrument used is a four-tier diagnostic test, totaling 80 items with 5 anchor items, which are divided into several questions packages. The test subjects used were 300 first grade high school students in Yogyakarta Province. The FTDT content validity test involved 7 validators and analyzed using Aiken’s V. The validity is proven by the FTDT items fit with the Partial Credit Model (PCM). The analysis used is descriptive analysis and item characteristics with the PCM approach to obtain reliability estimates, difficulty level’s item estimates (bi) and participants’ ability estimates (θ) to determine the test information function value. Then, comparing the results of the weighting of 8 categories with 16 categories. The results show that (i) The FTDT construction consists of 4 tiers consisting of: questions, level of confidence in answering questions, reasons and level of confidence in choosing reasons totaling 25 questions with 5 anchor items based on 4 aspects and 5 sub-aspects, contains senior high school of physics topics for first class 2nd Semester, which is used to measure students’ conceptual understanding of physics (ii) the FTDT scoring system is categorized into 8 categories more suitable than 16 categories, (iii) The overall FTDT items fit the PCM model and the items has a difficulty level in the range between -2, 0 to 2.0, indicating that the scoring system between 0 and 7 accurately distributes normal symmetrical difficulty level ranging from low to high difficulty. This FTDT is actually able to explore and diagnose the weaknesses and misconceptions of students more deeply and can be used as a reference to determine the causes of misconceptions, as well as a reference for determining learning in learning to reduce students’ misconceptions. In addition, these results are expected to contribute to the scoring of four-tier format test instruments in the field of physics.