Mapping Test Items to the 1992 NAEP Mathematics Achievement Level Descriptions: Mathematics Educators’ Interpretations and Their Relationship to Student Performance

A study by CRESST researchers concludes that the NAEP descriptions of mathematics achievement levels of basic, proficient, and advanced, do not match what students are really able to do. Independent judges analyzed the specific skills associated with each achievement level and the appropriate NAEP test items. “The lack of agreement, conclude the researchers, “among the mathematics educators indicates that the criteria to which NAEP is attempting to relate performance are too ill-defined.” According to the researchers, judges reported significant problems with the clarity of the achievement level descriptions. One example was the descriptor used to describe basic and proficient levels at the 4th-grade level related to student skills with whole numbers. The basic descriptor said, “estimate with whole numbers,” while the advanced descriptor said, “[use] whole numbers to estimate.” “…I felt that some descriptors were intentionally ambiguous,” said one of the judges. The study sought answers to three questions, (1) how clear were the achievement level descriptions, (2) was NAEP assessing the knowledge and skills that characterize the expectations of standards embodied in the achievement levels descriptions, and (3) could students actually do what the achievement levels said they should be able to do? “It is not appropriate to conclude,” write the authors, “that students scoring at particular levels know and are able to do what NAGB’s achievement level descriptions indicate they should be able to do.” The researchers recommend that the National Assessment Governing Board, who has responsibility for the NAEP, start the process anew by closely aligning the achievement levels with the development of new assessment frameworks and items.