Does the Time between Scoring Sessions Impact Scoring Accuracy? An Evaluation of Constructed-Response Essay Responses on the "GRE"® General Test. Research Report. ETS RR-18-31

This report investigates whether the time between scoring sessions has an influence on operational and nonoperational scoring accuracy. The study evaluates raters' scoring accuracy on constructed-response essay responses for the "GRE"® General Test. Binomial linear mixed-effect models...

Full description

Saved in:
Bibliographic Details
Published inETS research report series
Main Authors Finn, Bridgid, Wendler, Cathy, Ricker-Pedley, Kathryn L, Arslan, Burcu
Format Journal Article
LanguageEnglish
Published Educational Testing Service 01.12.2018
Subjects
Accuracy
College Entrance Examinations
Conceptual Tempo
Essay Tests
Evaluators
Graduate Study
Interrater Reliability
Intervals
Predictive Validity
Scoring
Time
Online AccessGet full text

Cover

More Information
Summary:This report investigates whether the time between scoring sessions has an influence on operational and nonoperational scoring accuracy. The study evaluates raters' scoring accuracy on constructed-response essay responses for the "GRE"® General Test. Binomial linear mixed-effect models are presented that evaluate how the effect of various predictors, such as time spent scoring each response, days in a scoring gap, and number of consecutive days of scoring, relate to scoring accuracy. Results suggest that for operational scoring, the number of days in a scoring gap has a negative influence on performance. The findings, as well as other results from the models, are discussed in the context of cognitive influences on knowledge and skill retention.
ISSN:2330-8516