Emotional feature guidance-based speaking head video generation method

• Main Authors: HE ZHITONG, YANG ZHIJING, KE ZIMING, HUANG YANCONG, FU CHENBO, FAN YAOZHOU, CHEN TIANSHUI
• Format: Patent
• Language: Chinese; English
• Published: 09.05.2023
• Subjects: ACOUSTICS; CALCULATING; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS; COMPUTING; COUNTING; IMAGE DATA PROCESSING OR GENERATION, IN GENERAL; MUSICAL INSTRUMENTS; PHYSICS; SPEECH ANALYSIS OR SYNTHESIS; SPEECH OR AUDIO CODING OR DECODING; SPEECH OR VOICE PROCESSING; SPEECH RECOGNITION

The invention discloses an emotional feature guidance-based speaking head video generation method, which introduces emotional feature guidance, a model obtained by training can have an emotional expression ability, a generated speaking head can have rich emotional vivid expression, and the problem of neutral solid-state expression of the current speaking head can be well solved. Meanwhile, through the three-dimensional reconstruction technology based on the two-dimensional data, approximate data are obtained from an existing two-dimensional data set with rich emotions, and the problem of lack of data can be solved. The method focuses on improving the emotional ability expressed by the character image video while generating the character image video. While the propagation content is ensured, the character in the video is endowed with the emotion tension matched with the propagation content, so that the character expression in the video has better emotion characteristics, the authenticity of a virtual image is