Long-Term Impact of a Faculty Development Program on Student Evaluations of Teaching

• Published in: Association for Engineering Education - Engineering Library Division Papers
• Main Authors: Kerst, Julia F, Pfershy, Hanna, DeMonbrun, Robert Matthew, Finelli, Cynthia J
• Format: Conference Proceeding
• Language: English
• Published: Atlanta American Society for Engineering Education-ASEE 24.06.2017
• Subjects: Active learning; Colleges & universities; Community participation; Engineering education; Evaluation; Learning; Regression analysis; Slopes; Students; Teacher evaluations; Teachers; Teaching

Despite recent research on the use of active learning and its benefits in the classroom, many engineering faculty have yet to implement these practices. There are many factors for this lack of adoption, including lack of familiarity with the practices, content and course restrictions, and negative course evaluations that might affect tenure. At the University of Michigan, we have developed a faculty learning community program to address some of these common barriers. Faculty participants in the program meet monthly throughout one semester to discuss active learning pedagogies and strategies for successfully adopting them. In order to assess the impact of the program, we have compiled student evaluation data for faculty who applied to participate in the program. Student evaluations have been shown to be a particularly influential motivator in faculty decisions to adopt active learning pedagogies, and can provide substantial feedback on how students view these practices. Our study consisted of 81 engineering faculty members of differing departments, genders, and ranks. Of these, 40 faculty had participated in the faculty learning community program (the “intervention” group). The other 41 had applied to participate but, due to program limitations, were not accepted to the program (the “control” group). Students in every course were invited to complete an end-of-semester student evaluation of teaching form. The evaluation included four standard items and other items that were selected by the college, department, and/or individual instructor. For the 81 faculty in this study, we analyzed student evaluations data for all academic terms where faculty taught a course from Fall 2008 through Winter 2016. We excluded evaluations for all graduate courses, independent studies, and all courses with fewer than five responses. We then identified 12 items from the student evaluation data that were both relevant and had a substantial number of responses. Next, for every academic term in which the instructor taught, we computed an item-by-item course average for each instructor, and then we compared those responses with previous and follow-up terms to identify the impact of participating in the program. Using ordinary least squares (OLS) regression, we studied the linear relationship between time (e.g., terms before and after the instructor applied to participate in the faculty learning community) and the average score for each of the 12 items. On average, we have student evaluation data for 10 terms before the instructors applied and 8 terms after. When comparing the models of the control and intervention groups, we found that the slopes for our control group in the pre- and post-application periods remained stagnant or slightly declined, whereas the slopes for the intervention groups after program participation steadily increased. The slopes of the intervention group improved in 10 out of 12 cases, while only 3 of the control slopes improved. The full paper will further explore these preliminary findings as well as other statistics and will provide a summary of the program’s effectiveness.