Referees or sponsors? The role of evaluators in the promotion of research scientists in a public research organization

• Published in: Research evaluation Vol. 28; no. 1; pp. 63 - 72
• Main Authors: Glennie, Miriam, O’Donnell, Michael, Brown, Michelle, Benson, John
• Format: Journal Article
• Language: English
• Published: Oxford University Press 01.01.2019
• Subjects: Case Studies; Classification; College Faculty; Context Effect; Decision Making; Evaluators; Focus Groups; Foreign Countries; Institutional Evaluation; Performance Based Assessment; Research Universities; Researchers; Role; Specialization; State Universities; Australia; researcher evaluation; sponsorship; typology; public research organization; promotion
• ISSN: 0958-2029; 1471-5449
• DOI: 10.1093/reseval/rvy035

Abstract Evaluators play a central role in assessments of researchers’ performance for reward, but the nature of their role and influence is not well understood. Ongoing reliance on evaluator judgement is typically justified as a need for referees in contests for reward, because quantitative performance measures alone can be subject to distortion. Yet, if evaluators are able to privately establish or interpret the performance standards utilized in evaluation, it may inhibit equality of opportunity, limit applicant pools, and reinforce existing inequalities. This article untangles the different roles played by evaluators through the development of a typology of systems of performance evaluation from existing literature. The typology is then applied to one type of evaluation system—promotion in a public research organization (PRO)—to investigate how and why particular evaluator roles emerge, and what contextual factors influence their implementation. Data are drawn from a mixed-method case study of a large Australian PRO, which includes data from one focus group, 22 in-depth interviews and 803 survey responses. The case study finds that evaluators’ role can extend to sponsorship of researchers through the promotion system when diversity in research specializations inhibits the establishment of uniform performance standards. It also demonstrates that reporting lines and evaluator workload impact how and to whom sponsoring support is given.