Mixed methods analysis strategies in program evaluation beyond “a little quant here, a little qual there”

• Published in: Journal of engineering education (Washington, D.C.) Vol. 108; no. 2; pp. 178 - 196
• Main Authors: Reeping, David, Taylor, Ashley R., Knight, David B., Edwards, Cherie
• Format: Journal Article
• Language: English
• Published: Hoboken, USA John Wiley & Sons, Inc 01.04.2019; Wiley Periodicals, Inc; Blackwell Publishing Ltd
• Subjects: analysis strategies; Colleges & universities; Data acquisition; Data Analysis; Education; Engineering; Engineering Education; Mixed Methods Research; Program Evaluation; Qualitative analysis; research approaches; Research Design; Taxonomy
• ISSN: 1069-4730; 2168-9830
• DOI: 10.1002/jee.20261

Background Mixed methods research designs in engineering education often frame the “mixed” aspect of the design from the perspective of data collection. However, intentional mixing throughout the research design—particularly during data analysis—may enable richer meta‐inferences about a phenomenon. Purpose This paper provides examples of strategies for mixing during data analysis in mixed methods program evaluation in engineering education. Although the context is program evaluation, the procedures are applicable in engineering education research more broadly. Scope/Method This review presents examples of mixed methods analysis strategies in the context of a data set with qualitative and quantitative data from a global engineering program at a large Mid‐Atlantic university. The structure of the examples presents mixed research questions, pragmatic purposes for such studies, and examples of different mixing strategies for the analysis stage. The mixing strategies highlighted include extreme case sampling, converting, creating a blended variable, blending variables and themes across strands, and cross‐case comparison. Conclusions This review of mixed methods in program evaluation prompted a reflection of processes in the design of studies, how the designs are described beyond the usual taxonomies in the mixed methods literature, and implications for the greater community of engineering education researchers. Five mixed methods designs were formulated around the mixing strategies.