Speech translation model modeling method and device based on speech synthesis data

• Main Authors: YANG MURUN, DU QUAN
• Format: Patent
• Language: Chinese; English
• Published: 21.03.2023
• Subjects: ACOUSTICS; CALCULATING; COMPUTING; COUNTING; ELECTRIC DIGITAL DATA PROCESSING; MUSICAL INSTRUMENTS; PHYSICS; SPEECH ANALYSIS OR SYNTHESIS; SPEECH OR AUDIO CODING OR DECODING; SPEECH OR VOICE PROCESSING; SPEECH RECOGNITION

The invention relates to a speech translation model modeling method and device based on speech synthesis data, and belongs to the technical field of natural language processing. The problem that in the prior art, a speech translation model is small in training data size and insufficient in utilization, and consequently the translation result is inaccurate is solved. The modeling method comprises the following steps: acquiring a general speech synthesis data set, and training to obtain a general speech synthesis model; acquiring a speech translation data set of the target domain; performing fine tuning on the universal speech synthesis model by using the speech translation data set to obtain a special speech synthesis model; inputting the source language annotation text into a special voice synthesis model, and generating a plurality of pieces of voice synthesis pseudo data according to a preset proportion to obtain a pseudo voice data set; and constructing an initial speech translation model, training the ini