A multimodal discrimination method for the response to name behavior of autistic children based on human pose tracking and head pose estimation

• Published in: Displays Vol. 76; p. 102360
• Main Authors: Song, Chunyi, Wang, Shigang, Chen, Meimei, Li, Honghua, Jia, Feiyong, Zhao, Yunxiu
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.01.2023
• Subjects: Autism spectrum disorder (ASD); Early screening; Head pose estimation; Human pose tracking; Response to name; Response to name; Early screening; Human pose tracking; Autism spectrum disorder (ASD); Head pose estimation
• DOI: 10.1016/j.displa.2022.102360
• ISSN: 0141-9382

The incidence rate of autism spectrum disorder (ASD) has been rising in the past decades, and has reached 1 ∼ 2%. Clinical ASD screening requires long testing time, complicated doctor training and high cost. Therefore, an objective and accurate automated method is urgently needed for large-scale early screening of autistic children. The lack of response to one’s own name is one of the early warning signals of ASD. Response to name is included in various ASD early screening and diagnostic assessments. In this paper, we propose an algorithm that combines automatic name detection with human pose tracking and head pose estimation in computer vision. The final prediction is determined by the combination of shoulder rotation angle, head pose information and duration. The experimental data comes from the process of clinical screening for ASD, which has less laboratory control, more realistic child response, and a flexible number of accompanying parents. Thirty subjects including 29 children with ASD and 1 typical developing (TD) child participated in the experiment. The experimental results show that the consistency between our proposed method and clinical diagnosis achieves 93.3%, which indicates our method can play an effective auxiliary role in the early screening of ASD. •A multi perspective platform is established to capture ASD children’s behavior patterns.•A “Response to name Dataset” for autism spectrum disorder is established.•Body pose and head pose are combined to judge children’s RTN reaction.