Limited-Information Goodness-of-Fit Testing of Diagnostic Classification Item Response Theory Models

Although diagnostic classification models have become increasingly popular in educational and psychological measurement, one noted limitation has been a lack of established methods for evaluating their goodness-of-fit. This is a significant problem, since inferences made on the basis of these models (including classification of examinees) may be invalid when the models are badly misspecified. This study examines the potential utility of two indices for testing the fit of these models: Maydeu-Olivares and Joe’s (2006) M2 and Chen and Thissen’s (1997) local dependence (LD) chi-square. These two indices have been previously applied to standard item response theory models. Here we evaluate their performance when applied to diagnostic classification models. Both were found to have good calibration under a wide range of data generating conditions and were sensitive to some–but not all–types of misspecification examined. The study suggests that M2 and the LD index may be quite useful for detecting and characterizing diagnostic classification model misfit.