Publications & Resources

A New Statistic for Evaluating Item Response Theory Models for Ordinal Data

Mar 2014

Li Cai and Scott Monroe

We propose a new limited-information goodness of fit test statistic C2 for ordinal IRT models. The construction of the new statistic lies formally between the M2 statistic of Maydeu-Olivares and Joe (2006), which utilizes first and second order marginal probabilities, and the M2* statistic of Cai and Hansen (2013), which collapses the marginal probabilities into means and product moments. Unlike M2*, C2 may be computed even when the number of items is small and the number of categories is large. It is as well calibrated as the alternatives and can be more powerful than M2. When all items are dichotomous, C2 becomes equivalent to M2*, which is also equivalent to M2. We analyze empirical data from a patient-reported outcomes measurement development project to illustrate the potential differences in substantive conclusions that one may draw from the use of different statistics for model fit assessment.

Cai, L. & Monroe, S. (2014). A new statistic for evaluating item response theory models for ordinal data (CRESST Report 839). Los Angeles: University of California, Los Angeles, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).