Interim Report: The Reliability of Vermont Portfolio Scores in the 1992-93 School Year

This interim report provides the first results of the second year of the Vermont assessment program, focusing on program implementation, effects on education, and the quality of performance data. “The program was altered in many ways in 1992-93,” wrote the CRESST/RAND research team, “which resulted in a clear increase in the reliability with which mathematics portfolios were scored.” However, the researchers added that while this progress is encouraging, scoring reliability in mathematics needs to be increased further if the program goals are to be achieved. Refining or simplifying scoring rubrics and placing further restrictions on types of tasks considered acceptable for inclusion in mathematics portfolios, are among the types of clarifications that may result in increased reliability. “In contrast,” said the researchers, “the reliability of writing portfolio scores did not improve substantially and was considerably lower than in mathematics.” The researchers believe that it is unrealistic to expect a substantial rate of improvement in the reliability of the writing portfolio scores unless the program itself is substantially revised.