Application of Generalizability Theory to Concept-Map Assessment Research

In the first part of this paper we discuss the feasibility of using Generalizability (G) Theory (see Footnote 1) to examine the dependability of concept map assessments and to design a concept map assessment for a particular practical application. In the second part, we apply G theory to compare the technical qualities of two frequently used mapping techniques: construct-a-map with created linking phrases (C) and construct-a-map with selected linking phrases (S). We explore some measurement facets that influence concept-map scores and estimate how to optimize different concept mapping techniques by varying the conditions for different facets. We found that C and S were not technically equivalent. The G coefficients for S were larger than those for C. Furthermore, a D study showed that fewer items (propositions) would be needed for S than C to reach desired level of G coefficients if only one occasion could be afforded. Therefore, S might be a better candidate than C in the large-scale summative assessment, while C would be preferred as a formative assessment in classroom.