Thermal Facial Analysis for Deception Detection

• Published in: IEEE transactions on information forensics and security Vol. 9; no. 6; pp. 1015 - 1023
• Main Authors: Rajoub, Bashar A., Zwiggelaar, Reyer
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.06.2014; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Accuracy; Applied sciences; Artificial intelligence; Computer science; control theory; systems; Data processing. List processing. Character string processing; Deception; Exact sciences and technology; Facial; Feature extraction; Forensic engineering; Heat detection; Imaging; Interviews; Mathematical models; Memory organisation. Data processing; Methodology; Monitors; Pattern recognition. Digital image processing. Computational geometry; Polygraphs; Robustness; Signatures; Software; Stress; Thermal analysis; Thermal imaging; Automated deception detection; behavioural analysis; thermal imaging; facial analysis; Infrared thermography; Nearest neighbour; Computer vision; Methodology; Face recognition; Decision making; Cross validation; Fraud; Generalization; Modeling; Lying; Behavioral analysis; Automatic measurement; Classification; Thermal analysis; Thermal imaging; Thermal test
• ISSN: 1556-6013; 1556-6021
• DOI: 10.1109/TIFS.2014.2317309

Thermal imaging technology can be used to detect stress levels in humans based on the radiated heat from their face. In this paper, we use thermal imaging to monitor the periorbital region's thermal variations and test whether it can offer a discriminative signature for detecting deception. We start by presenting an overview on automated deception detection and propose a novel methodology, which we validate experimentally on 492 thermal responses (249 lies and 243 truths) extracted from 25 participants. The novelty of this paper lies in scoring a larger number of questions per subject, emphasizing a within-person approach for learning from data, proposing a framework for validating the decision making process, and correct evaluation of the generalization performance. A k -nearest neighbor classifier was used to classify the thermal responses using different strategies for data representation. We report an 87% ability to predict the lie/truth responses based on a within-person methodology and fivefold cross validation. Our results also show that the between-person approach for modeling deception does not generalize very well across the training data.