Big Data for Little Kids
For the past four years, CRESST has collaborated with PBS KIDS on their Ready To Learn (RTL) program. Sponsored by the U.S. Department of Education, this program leverages content on television, the Internet and other media to help millions of young children—particularly those living in poverty—learn the basic reading, math and science skills needed for success.
A major part of RTL is their Learning Analytics Platform (LAP), which captures student interactions across all PBS KIDS learning technologies, such as games, classroom-based smart whiteboards, and mobile/desktop platforms. Leveraging our unique expertise in educational assessment, learning and psychometrics—as well as a background in game design and game data analysis—we were able to design and implement a telemetry format to measure and analyze the learning processes of these children.
Our team, led by Eva Baker along with Greg Chung, Kilchan Choi and Charlie Parks, have analyzed this data to determine the instructional, assessment and interactive properties for each game. And the telemetry data these games generate can be massive. For example, one game can have hundreds of telemetry points and produce thousands of events in a single session. In fact, the LAP processes approximately 200GB of data from tens of thousands of players monthly.
As we continue our work with PBS KIDS through 2021, we will continue our mission to advance the state-of-the-art in the assessment and measurement of interactive experiences in support of children’s learning.
Navy Skill Decay Analysis
The U.S. Navy has identified that certain essential skills pertaining to electronics and troubleshooting deteriorates among some sailors between the time they complete “A” School (which teaches basic naval skills), and the time they enter “C” School (which teaches more advanced skills).
To address this situation, CRESST has been working with the Navy to uncover new ways to detect, analyze and remedy this decay in skills. Specifically, our work examines the retention of applied problem-solving and analytical skills over the course of a sailor’s training regime, which helps make their training more effective.
Our team, led by Eva Baker along with Alan Koenig, John Lee, Markus Iseli and Barbara Jones, have created a series of simulation-based assessments to measure and track the applied skills and knowledge that should be developed within specific training contexts. Our findings are not only helping to guide enhancements within the Navy’s training curriculum, they are also essential in developing optimal remediation pathways for students showing signs of skill degradation.
This latest effort is just one of more than a dozen Navy-funded projects we have taken the lead on, including the areas of detection, measurement and the automated assessment of cognitively complex, high-stakes performance. Prior domains have included Shipboard Damage Control, Tactical Planning & Implementation, Ship Navigation & Conning, and Rifle Marksmanship (within the Marine Corps).
Integration of Simulation-Based Measures and Models for Ultrasonography Assessment
Working with Pelagique, LLC, a Department of Defense medical contractor, CRESST has been instrumental in developing and validating ultrasound proficiency metrics—a critical step toward responsibly integrating ultrasound technology into clinical medicine.
As part of our pioneering work, we created an ontology of the fundamentals of ultrasound, which defines the knowledge, skills and user attributes required to: 1) measure user performance; 2) make inferences about their knowledge and skills based on performance over time; and 3) recommend targeted training based on these inferences.
We are currently creating an ontology-based assessment and training framework built on study results and expert feedback that includes management and analysis tools. These employ ontologies and related graphical probabilistic computational models.
This groundbreaking effort will be highly useful to future national and international professional organizations that monitor and guide ultrasound practice, licensure, accreditation and policy, offering the potential reduce costs and increase ultrasound accessibility for the entire medical industry.