An Approach to Analyzing the Cognitive Complexity of Science Performance Assessments

Psychological theories that describe the development of subject-matter competence in terms of changes in the quality of cognition provide a basis for reconsidering the design and evaluation of alternative assessments. Such an evaluation explicitly considers the thinking and reasoning activities elicited in assessment situations and the extent to which these activities are given preference in defining subject-matter achievement. This paper describes an approach to address this cognitive validity issue that utilizes comparative studies of expertise. Illustrative examples from cognitive analyses of current assessment practices make apparent the utility of this approach for identifying performance objectives and examining the correspondence among test objectives, performance scores, and observed cognitive activity. This approach calls for the integration of assessment practice and knowledge of learning, and challenges the measurement community to (a) reconceptualize achievement test theory and design to systematically incorporate the cognitive aspects of performance, and (b) formulate appropriate methodologies for analyzing the cognitive properties of assessments in various subject-matter areas.