Development of a situational judgment test for global engineering competency

• Published in: Journal of engineering education (Washington, D.C.) Vol. 109; no. 3; pp. 470 - 490
• Main Authors: Jesiek, Brent K., Woo, Sang Eun, Parrigon, Scott, Porter, Caitlin M.
• Format: Journal Article
• Language: English
• Published: Hoboken, USA John Wiley & Sons, Inc 01.07.2020; Wiley; Blackwell Publishing Ltd
• Subjects: assessment; College Students; Competence; Cultural Awareness; Cultural Context; Engineering; Engineering Education; Engineering profession; Engineers; Foreign Countries; global engineering; Globalization; intercultural competence; Quality assessment; Ratings; situational judgment test; Situational Tests; Student Evaluation; survey; Test Construction; Test Reliability; Test Validity; Validity; Work Experience; China
• ISSN: 1069-4730; 2168-9830
• DOI: 10.1002/jee.20325

Background As globalization continues to impact the engineering profession, many programs aim to prepare current and future engineers to work across national and cultural boundaries. Yet there remains a lack of quality tools for assessing global competency among engineers and other technical professionals, including their behavioral tendencies in global work situations. Purpose We introduce development of a situational judgment test (SJT) covering three dimensions of global engineering competency (GEC) in Chinese national/cultural context. The main aim of this article is to describe how the SJT was developed through a systematic multistep process. Secondarily, we explore relationships between SJT performance and other theoretically relevant variables. Methods After generating a large initial pool of SJT scenarios and behavioral response items, we used ratings from subject matter experts (SMEs) to select six SJT scenarios and create scoring keys for 26 response items. To further explore the instrument's validity, we deployed the SJT items, other relevant measures, and a demographic survey to a sample of practicing engineers (n = 400). Results SME ratings provide strong evidence for the content relevance of the GEC–SJT tool. Survey results also suggest positive relationships between SJT performance and Chinese cultural knowledge, age, and years of work experience. However, more validity and reliability evidence is needed before recommending wider use of the instrument. Conclusions Our findings suggest the SJT format as a promising behavior‐based approach to measuring global competency and other professional attributes in engineering. We also discuss directions for future research and training efforts related to assessing and developing global competency.