Houston, We Have a Problem: Teachers Find No Value in the SAS Education Value-Added Assessment System (EVAAS®)

• Published in: Education policy analysis archives Vol. 22; no. 98; pp. 98 - 139
• Main Author: Collins, Clarin
• Format: Journal Article
• Language: English
• Published: Colleges of Education at Arizona State University and the University of South Florida 27.10.2014; Arizona State University
• Subjects: accountability; education policy; Educational Assessment; Elementary School Teachers; Formative Evaluation; High Stakes Tests; Mixed Methods Research; Online Surveys; Reliability; Secondary School Teachers; Teacher Attitudes; Teacher Effectiveness; Teacher Evaluation; teacher quality; Teacher Surveys; United States (Southwest); Validity; Value-added models (VAMs)
• ISSN: 1068-2341; 1068-2341
• DOI: 10.14507/epaa.v22.1594

This study examined the SAS Education Value-Added Assessment System (EVAAS®) in practice, as perceived and experienced by teachers in the Southwest School District (SSD). To evaluate teacher effectiveness, SSD is using SAS EVAAS® for high-stakes consequences more than any other district or state in the country. A mixed-method design including a large-scale electronic survey was used to investigate the model’s reliability and validity; to determine whether teachers used the SAS EVAAS® data in formative ways as intended; to gather teachers’ opinions on SAS EVAAS®’s claimed benefits and statements; and to understand the unintended consequences that occurred as a result of SAS EVAAS® use in SSD. Results revealed that the reliability of the SAS EVAAS® model produced split and inconsistent results among teacher participants, and teachers indicated that students biased the SAS EVAAS® results. The majority of teachers disagreed with the company’s marketing claims and did not report similar SAS EVAAS® and principal observation scores, reducing the criterion-related validity of both measures of teacher quality. Many unintended consequences associated with the high-stakes use of SAS EVAAS® emerged through teachers’ responses, which revealed among others that teachers felt heightened pressure and competition, which reduced morale and collaboration, and encouraged cheating or teaching to the test in attempt to raise SAS EVAAS® scores. The results of this study, one of the first to investigate how the SAS EVAAS® model works in practice, should be considered by policymakers, researchers, and districts when considering implementing the SAS EVAAS®, or any value-added model for teacher evaluation.