Automated Scoring of Students’ Small‐Group Discussions to Assess Reading Ability
We explored the feasibility of using automated scoring to assess upper‐elementary students’ reading ability through analysis of transcripts of students’ small‐group discussions about texts. Participants included 35 fourth‐grade students across two classrooms that engaged in a literacy intervention c...
Saved in:
Published in | Educational measurement, issues and practice Vol. 37; no. 2; pp. 20 - 34 |
---|---|
Main Authors | , , , , , |
Format | Journal Article |
Language | English |
Published |
Washington
Wiley-Blackwell
01.06.2018
Wiley Subscription Services, Inc |
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | We explored the feasibility of using automated scoring to assess upper‐elementary students’ reading ability through analysis of transcripts of students’ small‐group discussions about texts. Participants included 35 fourth‐grade students across two classrooms that engaged in a literacy intervention called Quality Talk. During the course of one school year, data were collected at 10 time points for a total of 327 student‐text encounters, with a different text discussed at each time point. To explore the possibility of automated scoring, we considered which quantitative discourse variables (e.g., variables to measure language sophistication and latent semantic analysis variables) were the strongest predictors of scores on a multiple‐choice and constructed‐response reading comprehension test. Convergent validity evidence was collected by comparing automatically calculated quantitative discourse features to scores on a reading fluency test. After examining a variety of discourse features using multilevel modeling, results showed that measures of word rareness and word diversity were the most promising variables to use in automated scoring of students’ discussions. |
---|---|
Bibliography: | jelmore@lexile.com jagreene@email.unc.edu hburdickpersonal@gmail.com Jeffrey A. Greene, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599 pkm15@psu.edu cfiretto@gmail.com Audra E. Kosh, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, and MetaMetrics, Inc., 1000 Park Forty Plaza Drive, Suite 120, Durham, NC 27713 . Carla M. Firetto, The Pennsylvania State University, 201 Old Main, University Park, PA 16802 Hal Burdick, MetaMetrics, Inc., 1000 Park Forty Plaza Drive, Suite 120, Durham, North Carolina 27713 audrakosh@gmail.com Jeff Elmore, MetaMetrics, Inc., 1000 Park Forty Plaza Drive, Suite 120, Durham, NC 27713 P. Karen Murphy, The Pennsylvania State University, 201 Old Main, University Park, PA 16802 |
ISSN: | 0731-1745 1745-3992 |
DOI: | 10.1111/emip.12174 |