Dynamical Non-compensatory Multidimensional IRT Model Using Variational Approximation

• Published in: Psychometrika Vol. 88; no. 2; pp. 487 - 526
• Main Authors: Tamano, Hiroshi, Mochihashi, Daichi
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.06.2023; Springer Nature B.V
• Subjects: Algorithms; Assessment; Behavioral Science and Psychology; Computer Simulation; Humanities; Item response theory; Law; Monte Carlo Method; Psychology; Psychometrics; Skills; Software; Statistical Theory and Methods; Statistics for Social Sciences; Test Theory; Testing and Evaluation; Theory and Methods; item response theory; variational approximation; Kalman filter; knowledge tracing; linear dynamical systems
• ISSN: 0033-3123; 1860-0980; 1860-0980
• DOI: 10.1007/s11336-023-09903-y

Multidimensional item response theory (MIRT) is a statistical test theory that precisely estimates multiple latent skills of learners from the responses in a test. Both compensatory and non-compensatory models have been proposed for MIRT: the former assumes that each skill can complement other skills, whereas the latter assumes they cannot. This non-compensatory assumption is convincing in many tests that measure multiple skills; therefore, applying non-compensatory models to such data is crucial for achieving unbiased and accurate estimation. In contrast to tests, latent skills will change over time in daily learning. To monitor the growth of skills, dynamical extensions of MIRT models have been investigated. However, most of them assumed compensatory models, and a model that can reproduce continuous latent states of skills under the non-compensatory assumption has not been proposed thus far. To enable accurate skill tracing under the non-compensatory assumption, we propose a dynamical extension of non-compensatory MIRT models by combining a linear dynamical system and a non-compensatory model. This results in a complicated posterior of skills, which we approximate with a Gaussian distribution by minimizing the Kullback–Leibler divergence between the approximated posterior and the true posterior. The learning algorithm for the model parameters is derived through Monte Carlo expectation maximization. Simulation studies verify that the proposed method is able to reproduce latent skills accurately, whereas the dynamical compensatory model suffers from significant underestimation errors. Furthermore, experiments on an actual data set demonstrate that our dynamical non-compensatory model can infer practical skill tracing and clarify differences in skill tracing between non-compensatory and compensatory models.