Model training method and device and device for model training

The embodiment of the invention provides a model training method and device and a device for model training. The method comprises the steps: acquiring training data, wherein the training data comprise first training data irrelevant to a pre-training task and second training data relevant to the pre-...

Full description

Saved in:
Bibliographic Details
Main Author ZHAN JIQING
Format Patent
LanguageChinese
English
Published 25.02.2022
Subjects
CALCULATING
COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
COMPUTING
COUNTING
ELECTRIC DIGITAL DATA PROCESSING
HANDLING RECORD CARRIERS
PHYSICS
PRESENTATION OF DATA
RECOGNITION OF DATA
RECORD CARRIERS
Online AccessGet full text

Cover

More Information
Summary:The embodiment of the invention provides a model training method and device and a device for model training. The method comprises the steps: acquiring training data, wherein the training data comprise first training data irrelevant to a pre-training task and second training data relevant to the pre-training task; inputting the first training data into a language model, and pre-training the language model based on the pre-training task to obtain a candidate language model; and adjusting the candidate language model based on the second training data to obtain a target language model. The candidate language model obtained by the embodiment of the invention can be suitable for the pre-training task, and the network structure, parameters and the like better meet the task requirements of the pre-training task; and the candidate language model is adjusted based on the second training data related to the pre-training task, and the model parameters are adjusted according to the loss value of the candidate model, so th
Bibliography:Application Number: CN202111276228