Online versus paper evaluations: differences in both quantitative and qualitative data

• Published in: Journal of computing in higher education Vol. 24; no. 1; pp. 58 - 69
• Main Authors: Burton, William B., Civitano, Adele, Steiner-Grossman, Penny
• Format: Journal Article
• Language: English
• Published: Boston Springer US 01.04.2012; Springer; Springer Nature B.V
• Subjects: College Students; Comparative Analysis; Computation; Computer Uses in Education; Conversion; Course Evaluation; Data analysis; Data Collection; Education; Educational Resources; Educational Technology; Evaluation Methods; Evaluators; Evidence; Factor Analysis; Feedback (Response); Higher Education; Internet; Interrater Reliability; Learning and Instruction; Literature Reviews; Medical; Medical Education; Medical Schools; Medical Students; On-line systems; Online; Online Surveys; Online Systems; Printed Materials; Program Evaluation; Psychometrics; Qualitative Research; Questionnaires; Response Rates (Questionnaires); Statistical Analysis; Students; Validity; Student feedback; Course evaluation questionnaires; Inter-rater reliability; Factor analysis; Qualitative data
• DOI: 10.1007/s12528-012-9053-3
• ISSN: 1042-1726

This study sought to determine if differences exist in the quantitative and qualitative data collected with paper and online versions of a medical school clerkship evaluation form. Data from six-and-a-half years of clerkship evaluations were used, some collected before and some after the conversion from a paper to an online evaluation system. The quantitative data consisted of a composite score based on the average of several Likert-type items; the qualitative data consisted of open-ended comments about the clerkships. Clerkship ratings were more positive in the online version. Students made significantly longer comments about both strengths and weaknesses on the online form than on the paper form. In addition, comments made on the online form were judged to be more informative and showed less evidence of “negativity” than those made on the paper form. The findings suggest that both quantitative and qualitative data obtained with online evaluation forms can differ in important ways from data collected with paper forms.