Human and Automated Assessment of Oral Reading Fluency

• Published in: Journal of educational psychology Vol. 105; no. 4; pp. 1142 - 1151
• Main Authors: Bolaños, Daniel, Cole, Ron A, Ward, Wayne H, Tindal, Gerald A, Hasbrouck, Jan, Schwanenflugel, Paula J
• Format: Journal Article
• Language: English
• Published: American Psychological Association 01.11.2013
• Subjects: Accuracy; Automation; Classification; Comparative Analysis; Correlation; Interrater Reliability; National Assessment of Educational Progress; Oral Reading; Reading Fluency; Reading Rate; Reading Tests; Scoring; Student Evaluation
• ISSN: 0022-0663
• DOI: 10.1037/a0031479

This article describes a comprehensive approach to fully automated assessment of children's oral reading fluency (ORF), one of the most informative and frequently administered measures of children's reading ability. Speech recognition and machine learning techniques are described that model the 3 components of oral reading fluency: word accuracy, reading rate, and expressiveness. These techniques are integrated into a computer program that produces estimates of these components during a child's 1-min reading of a grade-level text. The ability of the program to produce accurate assessments was evaluated on a corpus of 783 one-min recordings of 313 students reading grade-leveled passages without assistance. Established standardized metrics of accuracy and rate (words correct per minute [WCPM]) and expressiveness (National Assessment of Educational Progress Expressiveness scale) were used to compare ORF estimates produced by expert human scorers and automatically generated ratings. Experimental results showed that the proposed techniques produced WCPM scores that were within 3-4 words of human scorers across students in different grade levels and schools. The results also showed that computer-generated ratings of expressive reading agreed with human raters better than the human raters agreed with each other. The results of the study indicate that computer-generated ORF assessments produce an accurate multidimensional estimate of children's oral reading ability that approaches agreement among human scorers. The implications of these results for future research and near term benefits to teachers and students are discussed.