Design of a Screen-Based Simulation for Training and Automated Assessment of Teamwork Skills

Assessment, Learning Design, Technology

While communication and teamwork skills are increasingly recognized as important factors in improving patient safety, training of teamwork skills (“team training”) is not routinely incorporated into graduate training or continuing medical education programs. Communication failures between team members are responsible for up to 70% of medical errors.

Traditional team training sessions have consisted of classroom-based didactic presentations and/or resource-intense simulator-based programs that require in person attendance and facilitated debriefing. Commercially available training products (such as simulators and simulation software) have not fully incorporated team training with automated assessment in a manner that can be distributed widely for deliberate practice. The recently developed TeamSTEPPS online course can reach a wider audience. However, challenges include: a mostly didactic (narrated PowerPoint slides), non-interactive interface; the lack of opportunity to apply skills; lack of repetitive practice and feedback; and limited follow-up to assess skills acquisition and retention.

In collaboration with the UCLA Simulation Center (David Geffen School of Medicine, Departments of General Surgery/Trauma and Anesthesiology), the UCLA Graduate School of Education and Information Sciences (GSE&IS), and the UCLA Center for Advanced Surgical and Interventional Technology (CASIT), CRESST is currently developing simulation-based assessment and instruction of teamwork skills in the medical domain: Scenarios for the emergency room, the operating room, and the intensive care unit have been designed and are currently integrated into a screen-based simulation, which will incorporate an automated assessment engine capable of real-time analysis of observable learner actions. The assessment engine will make inferences of learner proficiency based on observed behavior by using computational graphical models.

The simulation will be accessible online (via secure server) or offline, will permit repetitive practice, will be engaging (game-like), and will provide feedback as formative assessment after each scenario module. The teamwork skills addressed – leadership, situation monitoring, communication, mutual support, and psychological safety – are relevant to healthcare in general, and particularly to military healthcare, where deployments often disrupt designated teams and heighten the need to form cohesive units quickly. Individual practice in synthetic training environments such as this one can supplement face-to-face team training.

Project Director(s):

Markus Iseli, Alan Koenig, John Lee