An EM-Based Method for Q-Matrix Validation

• Published in: Applied psychological measurement Vol. 42; no. 6; pp. 446 - 459
• Main Authors: Wang, Wenyi, Song, Lihong, Ding, Shuliang, Meng, Yaru, Cao, Canxi, Jie, Yongjing
• Format: Journal Article
• Language: English
• Published: Los Angeles, CA SAGE Publications 01.09.2018
• Subjects: Q-matrix; reduced RUM; cognitive diagnosis; DINA model; fraction-subtraction data; EM algorithm
• ISSN: 0146-6216; 1552-3497; 1552-3497
• DOI: 10.1177/0146621617752991

With the purpose to assist the subject matter experts in specifying their Q-matrices, the authors used expectation–maximization (EM)–based algorithm to investigate three alternative Q-matrix validation methods, namely, the maximum likelihood estimation (MLE), the marginal maximum likelihood estimation (MMLE), and the intersection and difference (ID) method. Their efficiency was compared, respectively, with that of the sequential EM-based δ method and its extension (ς2), the γ method, and the nonparametric method in terms of correct recovery rate, true negative rate, and true positive rate under the deterministic-inputs, noisy “and” gate (DINA) model and the reduced reparameterized unified model (rRUM). Simulation results showed that for the rRUM, the MLE performed better for low-quality tests, whereas the MMLE worked better for high-quality tests. For the DINA model, the ID method tended to produce better quality Q-matrix estimates than other methods for large sample sizes (i.e., 500 or 1,000). In addition, the Q-matrix was more precisely estimated under the discrete uniform distribution than under the multivariate normal threshold model for all the above methods. On average, the ς2 and ID method with higher true negative rates are better for correcting misspecified Q-entries, whereas the MLE with higher true positive rates is better for retaining the correct Q-entries. Experiment results on real data set confirmed the effectiveness of the MLE.