An Electronic Competency-Based Evaluation Tool for Assessing Humanitarian Competencies in a Simulated Exercise

• Published in: Prehospital and disaster medicine Vol. 32; no. 3; pp. 253 - 260
• Main Authors: Evans, Andrea B., Hulme, Jennifer M., Nugus, Peter, Cranmer, Hilarie H., Coutu, Melanie, Johnson, Kirsten
• Format: Journal Article
• Language: English
• Published: New York, USA Cambridge University Press 01.06.2017; Jems Publishing Company, Inc
• Subjects: Adult; Altruism; Benchmarking; Computer Simulation; Disaster relief; Disasters; Evaluation; Female; Health technology assessment; Humanitarian aid; Humans; Internet; Male; Original Research; Quebec; Reproducibility of Results; Rescue Work - manpower; Rescue Work - organization & administration; Rescue Work - standards; Simulation; Canada; Montreal Quebec Canada; United States > US; Quebec City Quebec Canada; evaluation; humanitarian response; humanitarian training; humanitarian workers; simulation exercise; competency; CBHA Consortium of British Humanitarian Agencies; SimEx simulation exercise; MD Medical Doctor
• ISSN: 1049-023X; 1945-1938
• DOI: 10.1017/S1049023X1700005X

The evaluation tool was first derived from the formerly Consortium of British Humanitarian Agencies' (CBHA; United Kingdom), now "Start Network's," Core Humanitarian Competency Framework and formatted in an electronic data capture tool that allowed for offline evaluation. During a 3-day humanitarian simulation event, participants in teams of eight to 10 were evaluated individually at multiple injects by trained evaluators. Participants were assessed on five competencies and a global rating scale. Participants evaluated both themselves and their team members using the same tool at the end of the simulation exercise (SimEx). All participants (63) were evaluated. A total of 1,008 individual evaluations were completed. There were 90 (9.0%) missing evaluations. All 63 participants also evaluated themselves and each of their teammates using the same tool. Self-evaluation scores were significantly lower than peer-evaluations, which were significantly lower than evaluators' assessments. Participants with a medical degree, and those with humanitarian work experience of one month or more, scored significantly higher on all competencies assessed by evaluators compared to other participants. Participants with prior humanitarian experience scored higher on competencies regarding operating safely and working effectively as a team member. This study presents a novel electronic evaluation tool to assess individual performance in five of six globally recognized humanitarian competency domains in a 3-day humanitarian SimEx. The evaluation tool provides a standardized approach to the assessment of humanitarian competencies that cannot be evaluated through knowledge-based testing in a classroom setting. When combined with testing knowledge-based competencies, this presents an approach to a comprehensive competency-based assessment that provides an objective measurement of competency with respect to the competencies listed in the Framework. There is an opportunity to advance the use of this tool in future humanitarian training exercises and potentially in real time, in the field. This could impact the efficiency and effectiveness of humanitarian operations. Evans AB , Hulme JM , Nugus P , Cranmer HH , Coutu M , Johnson K . An electronic competency-based evaluation tool for assessing humanitarian competencies in a simulated exercise. Prehosp Disaster Med. 2017;32(3):253-260.