Publications & Resources

Computer-Based Collaborative Knowledge Mapping to Measure Team Processes and Team Outcomes

Mar 1999

Harold F. O'Neil Jr., Gregory K. W. K. Chung, and Howard E. Herl

In this study we examined the feasibility and validity of using a computer-based, networked collaborative knowledge mapping system to measure teamwork skills. Student groups were assessed with our system twice in the same academic year, once in the fall and once in the following spring. Our study focused on the nature of the interaction between team members as they jointly constructed a knowledge map. Each student was randomly assigned to a team and communicated (anonymously) with other members by sending predefined messages. Teamwork processes were measured by examining message usage. Each message was categorized as belonging to one of six team processes: (a) adaptability, (b) communication, (c) coordination, (d) decision-making, (e) interpersonal, and (f) leadership. Team performance was measured by scoring each team’s knowledge map using four expert maps as the criterion. No significant correlation was found between the team processes and team outcomes. This unexpected finding may be due in part to a split-attention effect resulting from the design of the user interface. However, student teams were able to successfully construct knowledge maps, suggesting that our general approach to using networked computers to measure group processes remain viable given existing alternatives. In this study we examined the feasibility and validity of using a computer-based, networked collaborative knowledge mapping system to measure teamwork skills. Student groups were assessed with our system twice in the same academic year, once in the fall and once in the following spring. Our study focused on the nature of the interaction between team members as they jointly constructed a knowledge map. Each student was randomly assigned to a team and communicated (anonymously) with other members by sending pre-defined messages. Teamwork processes were measured by examining message usage. Each message was categorized as belonging to one of six team processes: (a) adaptability, (b) communication, (c) coordination, (d) decision-making, € interpersonal, and (f) leadership. Team performance was measured by scoring each team’s knowledge map using four expert maps as the criterion. No significant correlation was found between the team processes and team outcomes. This unexpected finding may be due in part to a split-attention effect resulting from the design of the user interface. However, student teams were able to successfully construct knowledge maps, suggesting that our general approach to using networked computers to measure group processes remain viable given existing alternatives.

O’Neil, H. F. Jr., Chung, G. K. W. K., & Herl, H. E. (1999). Computer-based collaborative knowledge mapping to measure team processes and team outcomes (CSE Report 502). Los Angeles: University of California, Los Angeles, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).