Automated Assessment of Domain Knowledge With Online Knowledge Mapping

A critical first step in developing training systems is gathering quality information about a trainee’s competency in a skill or knowledge domain. Such information includes an estimate of what the trainee knows prior to training, how much has been learned from training, how well the trainee may perform in future task situations, and whether to recommend remediation to bolster the trainee’s knowledge. This paper describes the design, development, testing, and application of a Web-based tool designed to assess a trainee’s understanding of a content domain in a distributed learning environment. The tool, called the CRESST Human Performance Knowledge Mapping Tool (HPKMT), enables trainees to express their understanding of a content area by creating graphical, network representations of concepts and links that define the relationships of concepts. We review and evaluate alternative knowledge mapping scoring methods and online mapping systems. We then describe the overall design approach, functionality, scoring method, usability testing, and authoring capabilities of the CRESST HPKMT. The paper ends with descriptions of applications of the HPKMT to military training, limitations of the system, and next steps.