Effects of robot gaze and voice human-likeness on users’ subjective perception, visual attention, and cerebral activity in voice conversations

• Published in: Computers in human behavior Vol. 141; p. 107645
• Main Authors: Li, Mingming, Guo, Fu, Wang, Xueshuang, Chen, Jiahao, Ham, Jaap
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.04.2023
• Subjects: Eye-tracking; fNIRS; Robot gaze; Subjective perception; Voice human-likeness; fNIRS; Robot gaze; Subjective perception; Eye-tracking; Voice human-likeness
• ISSN: 0747-5632
• DOI: 10.1016/j.chb.2022.107645

Robot gaze and voice are essential anthropomorphic features to promote users' engagement in voice conversations. Earlier research chiefly examined how robot gaze and voice human-likeness separately influenced users' subjective perception. When implementing gaze on robots with different human-like voices, there has little evidence of their possible interaction effects, particularly on users' visual attention and cerebral activity, which could help to understand the perceptual and cognitive processing of anthropomorphic features. Therefore, a within-subject experiment of voice conversations with diverse robot gaze (gaze versus no gaze) and human-like voices (high human-like versus low human-like) using subjective reporting, eye-tracker, and fNIRS was conducted. The results showed that the robot with gaze or a high human-like voice evoked more pleasure, higher arousal, more perceived likability, and less negative attitudes. Robot gaze significantly increased users' average fixation durations and total fixation time, while voice human-likeness prolonged first fixation durations. Moreover, the robot with a high human-like voice (or gaze) induced increased activity in the left DLPFC and decreased activity in the right Broca's area than that had no gaze (or a low human-like voice). The results suggest that robot gaze might chiefly capture users' sustained attention, voice human-likeness might attract users' initial attention, and they might jointly influence users' perceptual processing of prosodic features and emotional processing. •We examined the effects of robot gaze and voice human-likeness via subjective reporting, eye-tracking, and fNIRS experiment.•The robot with gaze or a high human-like voice captured users' higher subjective perception and diverse eye-tracking metrics.•Robot gaze and voice human-likeness jointly impacted users' HbO2 concentration changes in left DLPFC and right Broca's area.•Robot gaze might chiefly capture users' sustained attention, and voice human-likeness might attract users' initial attention.•They might jointly impact users' perceptual processing of prosodic features and emotional processing in cerebral activity.