Anesthesiology Skills Automated Assessment

Assessment, Learning Design, Technology

CRESST will work with a team from the UCLA Medical School and individuals from the UCLA Graduate School of Education and Information Studies to develop and research a medical simulation to assess teamwork skills. The simulation will be accessible online (via secure server) or offline, permit repetitive practice, be engaging (game-like), and provide feedback in the form of summative assessment after each scenario module. The skills to be addressed within the simulation are relevant to general aspects of healthcare and specifically to military healthcare, where deployments often disrupt designated teams and heighten the need to form cohesive units quickly. Individual practice in synthetic training environments such as this one can supplement face-to-face team training.

Our project objective is to research learning and performance differences between two primary modes to simulate team training: (1) an Evaluator Mode in which students interact with the simulation system by providing annotations on observed team interactions without affecting the outcome of the event; and (2) aGame-Play Mode in which students play the role of a team member (leader or follower) and contribute to the outcome of the simulated scenario. Both groups will receive identical instruction on one or more specific team training skills prior to each scenario, and both will receive summative feedback and scoring after each scenario.

A primary outcome of this research is focused on the training effectiveness differences between the two modes (evaluator vs. game-play). The reason for this comparison of training modes is to examine the development of synthetic team training from a cost-benefit perspective. This includes examining the development effort vs. learning and performance expectations. Our contentions are (1) that optimal team training relies on immersive participation which our game-play mode will emulate; and (2) that for team training, development of an evaluator-style simulation is less costly (regarding development time, programming expertise needed, resources required, etc.) compared to a game-play style simulator.