Pre-training model acquisition and application method and device, equipment, medium and product

• Main Authors: DONG CHUANLEI, XU CHUNYUAN, WANG LIFEI, JIA WEI, SHAO YEFEI, ZHANG ANJIE, CHEN ZIJIAN
• Format: Patent
• Language: Chinese; English
• Published: 06.05.2022
• Subjects: CALCULATING; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS; COMPUTING; COUNTING; ELECTRIC DIGITAL DATA PROCESSING; HANDLING RECORD CARRIERS; PHYSICS; PRESENTATION OF DATA; RECOGNITION OF DATA; RECORD CARRIERS

The embodiment of the invention discloses a pre-training model acquisition and application method and device, equipment, a medium and a product, and the pre-training model acquisition method comprises the steps: obtaining a plurality of pieces of training data, the training data comprising a training text; performing data processing on each character in the training text to obtain a character fusion vector of each character, the character fusion vector comprising a word vector of the character and at least one of the following feature vectors: a pronunciation feature vector, a stroke feature vector, a structure feature vector, a split character feature vector and a coding feature vector of the character; and performing self-supervised learning training on the initial pre-training model based on the character fusion vector of each character in the training text to obtain a target pre-training model. According to the technical scheme, the target pre-training model with relatively high recognition capability on