Erroneous detection of nonuniform DIF using the Breslow-Day test in a short test

• Published in: Quality & quantity Vol. 43; no. 1; pp. 35 - 44
• Main Authors: Aguerri, María Ester, Galibert, María Silvia, Attorresi, Horacio Félix, Prieto Marañón, Pedro
• Format: Journal Article
• Language: English
• Published: Dordrecht Springer Netherlands 2009; Springer; Springer Nature B.V
• Subjects: Binomial distribution; Contingency tables; Econometric models; History, theory and methodology; Item response theory; Logistics; Methodology; Methodology (Data Analysis); Methodology of the Social Sciences; Original Paper; Quantitative psychology; Regression analysis; Social Science Research; Social Sciences; Sociology; Logistic regression; Mantel–Haenszel procedure; Nonuniform differential item functioning; Three parameter logistic model; Breslow-Day test; Regression Analysis; Mantel-Haenszel procedure; Methodology
• ISSN: 0033-5177; 1573-7845
• DOI: 10.1007/s11135-007-9130-2

This paper studies the Type I error rate obtained using the Breslow-Day (BD) test to detect Nonuniform Differential Item Functioning (NUDIF) in a short test when the average ability of one group is significantly higher than that of the other. The performance is compared with the logistic regression (LR) and the standard Mantel-Haenszel procedure (MH). Responses to a 20-item test were simulated without Differential Item Functioning (DIF) according to the three-parameter logistic model. The manipulated factors were sample size and item parameters. The design yielded 40 conditions that were replicated 50 times and the false positive rate at a 5% significance level obtained with the three methods was recorded for each condition. In most cases, BD performed better than LR and MH in terms of proneness to Type I error. With the BD test, the Type I error rate was similar to the nominal one when the item with the highest discrimination and difficulty parameters in the case of equally sized groups was excluded from the goodness-of-fit to the binomial distribution (number of false positives among the fifty replications of a Bernoulli variable with parameter equal to 0.05).