Lip language recognition system and method based on ultrasonic perception and knowledge distillation

• Main Authors: YIN YAFENG, WANG ZHENG, LU SANGLU, XIE LEI
• Format: Patent
• Language: Chinese; English
• Published: 22.06.2021
• Subjects: CALCULATING; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS; COMPUTING; COUNTING; HANDLING RECORD CARRIERS; PHYSICS; PRESENTATION OF DATA; RECOGNITION OF DATA; RECORD CARRIERS

The invention discloses a lip language recognition system and method based on ultrasonic perception and knowledge distillation. The lip language recognition system comprises a data collection unit, a data preprocessing unit and a model training unit. The data collection unit comprises an ultrasonic data collection module and a visual data collection module; the data preprocessing unit comprises an ultrasonic preprocessing module and a visual preprocessing module; the model training unit comprises an ultrasonic lip language recognition module, a visual lip language recognition module and a knowledge distillation module; according to the invention, by using a knowledge distillation method, the limitation that only ultrasonic information is used for lip language recognition is made up, and the ultrasonic lip language recognition capability is enhanced. 本发明公开了一种基于超声波感知与知识蒸馏的唇语识别系统及方法，包括：数据收集单元，数据预处理单元，模型训练单元；数据收集单元包括：超声波数据收集模块，视觉数据收集模块；数据预处理单元包括：超声波预处理模块，视觉预处理模块；模型训练单元包括：超声波唇语识别模块，视觉唇语识别模块，知识蒸馏模块；本发明通过使用知识蒸馏的方法，弥