Detecting the Doubt Effect and Subjective Beliefs Using Neural Networks and Observers’ Pupillary Responses

• Published in: Neural Information Processing pp. 610 - 621
• Main Authors: Zhu, Xuanying, Qin, Zhenyue, Gedeon, Tom, Jones, Richard, Hossain, Md Zakir, Caldwell, Sabrina
• Format: Book Chapter
• Language: English
• Published: Cham Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Doubt; Information veracity; Neural networks; Pupillary responses; Semi-unsupervised training; Subjective belief; Trust
• ISBN: 9783030042110; 3030042111
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-030-04212-7_54

We investigated the physiological underpinnings to detect the ‘doubt effect’ – where a presenter’s subjective belief in some information has been manipulated. We constructed stimulus videos in which presenters delivered information that in some cases they were led to doubt, but asked to “present anyway”. We then showed these stimuli to observers and measured their physiological signals (pupillary responses). Neural networks trained with two statistical features reached a higher accuracy in differentiating the doubt/ manipulated-belief compared to the observers’ own veracity judgments, which is overall at chance level. We also trained confirmatory neural networks for the predictability of specific stimuli and extracted significant information on those stimulus presenters. We further showed that a semi-unsupervised training regime can use subjective class labels to achieve similar results to using the ground truth labels, opening the door to much wider applicability of these techniques as expensive ground truth labels (provenance) of stimuli data can be replaced by crowd source evaluations (subjective labels). Overall, we showed that neural networks can be used on subjective data, which includes observer perceptions of the doubt felt by the presenters of information. Our ability to detect this doubt effect is due to our observers’ underlying emotional reactions to what they see, reflected in their physiological signals, and learnt by our neural networks. This kind of technology using physiological signals collected in real time from observers could be used to reflect audience distrust, and perhaps could lead to increased truthfulness in statements presented via the Media.