Development and Psychometric Testing of the Rate and Accumulation Concept Inventory

• Published in: Journal of engineering education (Washington, D.C.) Vol. 107; no. 3; pp. 491 - 520
• Main Authors: Flynn, Carli D., Davidson, Cliff I., Dotger, Sharon
• Format: Journal Article
• Language: English
• Published: Hoboken, USA John Wiley & Sons, Inc 01.07.2018; Wiley Periodicals, Inc; Blackwell Publishing Ltd
• Subjects: Accumulation; Classical test theory; concept inventories; Diagnostic software; Diagnostic systems; Disproportionate Representation; Engineering Education; Evidence; Heat transmission; Item Response Theory; Mass flow; Mastery Learning; mathematics; Measures (Individuals); misconceptions; physical science; psychometric analysis; Psychometrics; Reliability analysis; Reliability engineering; Scientific Concepts; Structural Equation Models; Student Evaluation; Students; Test Construction; Test Reliability; Test Theory; Test Validity; Validity
• ISSN: 1069-4730; 2168-9830
• DOI: 10.1002/jee.20226

Background Because a fundamental understanding of rate and accumulation principles is important for engineering students across all subdisciplines, a method is needed to assess their understanding of these principles. Purpose/Hypothesis This article discusses the development of the Rate and Accumulation Concept Inventory (RACI) instrument and provides an analysis of its validity and reliability. The RACI is designed to test students’ (1) overall mastery of rate and accumulation concepts and (2) mastery of specific rate and accumulation concepts (e.g., mathematics, mass flow, and heat flow). Design/Methods Exploratory findings of students’ conceptual understanding prompted the development of a preliminary RACI instrument. Two pilot studies were conducted, with adjustments made to the instrument between each. Data from the second pilot were used to assess evidence of validity and reliability through classical test theory, item response theory, and structural equation modeling. Results Validity and reliability evidence indicates that the RACI can be used to measure students’ overall understanding of the concepts identified, though several items require revision or removal. Issues of potential construct underrepresentation were found in two of the conceptual categories. Conclusions The evidence of reliability and validity indicates the appropriateness of the RACI for assessing engineering students’ overall understanding of rate and accumulation principles. Additional research is needed to further enhance the validity and reliability of the RACI as a diagnostic tool, particularly in measuring students’ understanding of rate and accumulation concepts in particular physical contexts.