Beyond satisfaction scores: visualising student comments for whole-of-course evaluation

• Published in: Assessment and evaluation in higher education Vol. 46; no. 5; pp. 685 - 700
• Main Authors: Cunningham-Nelson, Samuel, Laundon, Melinda, Cathcart, Abby
• Format: Journal Article
• Language: English
• Published: Abingdon Routledge 19.07.2021; Taylor & Francis Ltd
• Subjects: College students; Course Evaluation; Feedback (Response); Focus groups; Health Sciences; Higher education; Interviews; Natural language processing; Polls & surveys; Qualitative research; Reliability; Semi Structured Interviews; sentiment analysis; Student Evaluation; Student Evaluation of Teacher Performance; student evaluation of teaching; Student Satisfaction; Student Surveys; Teacher evaluations; text analysis; Undergraduate Students; Validity; visualisation; Visualization
• ISSN: 0260-2938; 1469-297X
• DOI: 10.1080/02602938.2020.1805409

This study examines the utility of a new method of analysing and reporting qualitative student survey comments. Visualisations of the sentiment of and key themes from qualitative student survey comments were developed for 34 subjects from two undergraduate health sciences courses undergoing transformation. The course and subject visualisation reports were provided to academics who were members of the two course transformation teams. Utilising focus groups and semi-structured interview methodology, we examined academic perspectives on the potential usefulness of this new method of analysing and reporting qualitative comments from student evaluations of teaching (SET). Results indicate that visualisations are considered useful for focussing educators' attention on themes in qualitative comments rather than on individual negative comments. However, educators did not support use of the visualisations at this stage, citing concerns about the accuracy of sentiment analysis and the validity of SET surveys. There were many opportunities identified through the focus groups for future improvements to a visualisation technique for reporting student survey results.