Beyond traditional interviews: Psychometric analysis of asynchronous video interviews for personality and interview performance evaluation using machine learning

• Published in: Computers in human behavior Vol. 154; p. 108128
• Main Authors: Koutsoumpis, Antonis, Ghassemi, Sina, Oostrom, Janneke K., Holtrop, Djurre, van Breda, Ward, Zhang, Tianyi, de Vries, Reinout E.
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.05.2024
• Subjects: Algorithmic bias; Artificial intelligence; Asynchronous video interview; Personality; Trait activation theory; Asynchronous video interview; Algorithmic bias; Personality; Trait activation theory; Artificial intelligence
• ISSN: 0747-5632; 1873-7692
• DOI: 10.1016/j.chb.2023.108128

With the advent of new technology, traditional job interviews have been supplemented by asynchronous video interviews (AVIs). However, research on psychometric properties of AVIs is limited. In this study, 710 participants completed a mock AVI responding to eight personality questions (Extraversion, Conscientiousness). We collected self- and observer reports of personality, interview performance ratings, attractiveness, and AVI meta-information (e.g., professional attire, audio quality). Then, we automatically extracted the words, facial expressions, and voice characteristics from the videos and trained machine learning models to predict the personality traits and interview performance. Our algorithm explained substantially more variance in observer reports of Extraversion and Conscientiousness (average R2 = 0.32) and interview performance (R2 = 0.44), than self-reported Extraversion and Conscientiousness (average R2 = 0.12). Consistent with Trait Activation Theory, the explained variance in personality traits increased when participants responded to trait-relevant, compared to trait-irrelevant, questions. The test-retest reliability of our algorithm was somewhat stable over a time period of seven months, but lower than desired reliability standards in personnel selection. We examined potential sources of bias, including age, gender, and attractiveness, and found some instances of algorithmic bias (e.g., gender differences were often amplified in favor of women). •Machine learning explains personality/interview performance variance in AVIs.•Trait-relevant questions increase explained personality variance of AVIs.•Algorithmic assessment of AVIs is generally free from algorithmic bias.•Verbal, not facial/voice, features explain most personality variance.