Comparing Collaborative Problem Solving Profiles Derived from Human and Semi-Automated Annotation

New challenges in today's world have contributed to increased attention toward evaluating individuals' collaborative problem solving (CPS) skills. One difficulty with this work is identifying evidence of individuals' CPS capabilities, particularly when interacting in digital spaces. O...

Full description

Saved in:
Bibliographic Details
Published inGrantee Submission
Main Authors Andrews-Todd, Jessica, Steinberg, Jonathan, Pugh, Samuel L, D'Mello, Sidney K
Format Report
LanguageEnglish
Published 2022
Subjects
Automation
Comprehension
Cooperation
Documentation
Grade 7
Grade 8
Grade 9
Group Dynamics
Interaction
Interpersonal Communication
Problem Solving
Secondary School Students
Student Behavior
Student Participation
Teamwork
Online AccessGet more information

Cover

More Information
Summary:New challenges in today's world have contributed to increased attention toward evaluating individuals' collaborative problem solving (CPS) skills. One difficulty with this work is identifying evidence of individuals' CPS capabilities, particularly when interacting in digital spaces. Often human-driven approaches are used but are limited in scale. Machine-driven approaches can save time and money, but their reliability relative to human approaches can be a challenge. In the current study, we compare CPS skill profiles derived from human and semi-automated annotation methods across two tasks. Results showed that the same clusters emerged for both tasks and annotation methods, with the annotation methods showing agreement on labeling most students according to the same profile membership. Additionally, validation of cluster results using external survey measures yielded similar results across annotation methods. [This paper was published in: "CSCL2022 Proceedings," International Society of the Learning Sciences, 2022, pp. 363-366.]