Publications & Resources

Making Sense of Data From Complex Assessments

Mar 2001

Robert J. Mislevy, Linda S. Steinberg, Russell G. Almond, F. Jay Breyer, and Lynn Johnson

Advances in cognitive psychology deepen our understanding of how students gain and use knowledge, and broaden the range of performances and situations we want to see to acquire evidence about their developing knowledge. At the same time, advances in technology make it possible to capture more complex performances in assessment settings by including, as examples, simulation, interactivity, and extended responses. The challenge is making sense of the data that result. This presentation concerns as evidence-centered approach to the design and analysis of complex assessments. It presents a design framework that incorporates integrated structures for modeling knowledge and skills, designing tasts and extracting and synthesizing evidence. The ideas are illustrated in the context of a project with the Dental Interactive Simulation Corporation (DISC) in which problem solving in dental hygiene is assessed with computer-based simulations. After reviewing the substantiative grounding of this effort, we describe the design rationale, statistical and scoring models, and operational structures for the assessment protoype.

Mislevy, R. J., Steinberg, L. S., Almond, R. G., Breyer, F. J., & Johnson, L. (2001). Making sense of data from complex assessments (CSE Report 538). Los Angeles: University of California, Los Angeles, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).