Optimization method and device of large language model, electronic equipment and storage medium

• Main Authors: LU YAO, RHO KWANG-MYUNG, LIN FENG, MA XIAO
• Format: Patent
• Language: Chinese; English
• Published: 19.07.2024
• Subjects: CALCULATING; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS; COMPUTING; COUNTING; ELECTRIC DIGITAL DATA PROCESSING; PHYSICS

The invention discloses an optimization method and device of a large language model, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence, and the optimization method of the large language model specifically comprises the following steps: S10, generating a training data set in combination with an original LLM model and a large model SFT data set; s20, embedding a bypass network in a backbone network of the original LLM model to obtain a new LLM model, and inputting a mask lexical element sequence at an input end of the bypass network; s30, training the new LLM model on the training data set by adopting a loss function, and after training is completed, predicting a plurality of candidate lexical element sequences in one-time reasoning by the LLM model; and S40, the generation of the candidate lexical element sequence and the verification of the correctness of the candidate lexical element sequence are executed in parallel. The method has the beneficial effect