June 30, 2024
Calibration, Scoring, and Reporting at Multiple Levels of Granularity: Application to Alternate English Language Proficiency Assessment

Authors:
Yun-Kyung Kim & Li Cai
This report describes a calibration, scoring, and reporting method applicable to an assessment that requires the results to be generated at multiple levels of aggregation. The method is demonstrated using ELPA21’s Alternate English language proficiency assessment (Alt ELPA) as an example. Alt ELPA is an English language proficiency assessment designed to evaluate English language proficiency of English learners with the most significant cognitive disabilities. Alt ELPA is required to produce scores at three different levels of granularity: domain-level scores (Listening, Reading, Speaking, and Writing scores), modality-level scores (Receptive and Productive modality scores), and overall scores. In addition, due to the specificity of the target population (e.g., frequent domain exemptions), the sample size is relatively small, and the degree of missingness and incompleteness is relatively high, which poses an additional challenge to calibration.
Kim, Y., & Cai, L. (2024). Calibration, Scoring, and Reporting at Multiple Levels of Granularity: Application to Alternate English Language Proficiency Assessment (CRESST Report 875). UCLA/CRESST.
