Learning Complex Cognitive Skills With an Interactive Job Aid

Engineering Duty Officers (EDOs) in the U.S. Navy manage large development and procurement processes. Their initial training is provided in a six week EDO Basic Course at Port Hueneme, California. The students, who have higher degrees in one or more engineering disciplines, must learn to make complex decisions that incorporate the uncertainty of future events, and to convincingly present their acquisition recommendations to national leaders. Expected value theory provides one framework for making such complex decisions. Students compute an estimated value for each alternative choice by summing the utilities of all the potential consequences of that choice, and weighting those utilities by the estimated likelihood of that outcome. Using the iRides simulation-training system, we developed a software application that provides a simple interface for examining and presenting the expected values of choices in an EDO context. To support the use of this software tool-the EDO Decision Aid-in the context of the class, representations of specific alternative solutions to a class problem were developed and presented to student teams working on that problem. The EDO Decision Aid was designed to record student actions, including the selection of alternatives, the setting of utility values and estimated probabilities, and the setting of decision thresholds. Analysis of student actions allows us to compare both the time students spent using the tool to arrive at their final recommendation and the range of alternative solutions they considered. Lag Sequential Analysis then allows us to analyze the order of events in the student’s solution process. Two groups can be distinguished: those who approach the acquisition process from a global perspective; and those who considered utility and uncertainty to make a case-by-case determination of value. How instructors might use this information to make instructional changes is explored.