Using Regularized Methods to Validate Q-Matrix in Cognitive Diagnostic Assessment

• Published in: Journal of educational and behavioral statistics Vol. 50; no. 1; pp. 149 - 179
• Main Authors: Fu, Daoxuan, Qin, Chunying, Luo, Zhaosheng, Li, Yujun, Yu, Xiaofeng, Ye, Ziyu
• Format: Journal Article
• Language: English
• Published: Los Angeles, CA SAGE Publications 01.02.2025; American Educational Research Association
• Subjects: Cognitive Tests; Sample Size; Q-matrix; cognitive diagnosis; attribute; SCAD; Lasso
• ISSN: 1076-9986; 1935-1054
• DOI: 10.3102/10769986241240084

One of the central components of cognitive diagnostic assessment is the Q-matrix, which is an essential loading indicator matrix and is typically constructed by subject matter experts. Nonetheless, to a large extent, the construction of Q-matrix remains a subjective process and might lead to misspecifications. Many researchers have recognized the importance of estimating or validating the Q-matrix, but most of them focus on the conditions of relatively large sample sizes. This article aims to explore Q-matrix validation possibilities under small sample conditions and uses regularized methods to validate the Q-matrix based on the compensatory reparametrized unified model and generalized deterministic inputs, noisy “and” gate models. Simulation studies were conducted to evaluate the viability of the modified least absolute shrinkage and selection operator (Lasso) and modified smoothly clipped absolute deviation (SCAD) methods, comparing them with existing methods. Results show that the modified Lasso and the modified SCAD methods outperform the stepwise, Hull, and MLR-B methods in general, especially under the conditions of small sample sizes. While good recovery in all small sample size conditions is not guaranteed, the modified methods demonstrate advantages across various item quality conditions. Also, a real data set is analyzed to illustrate the application of the modified methods.