Distributed model training method and device

The embodiment of the invention provides a distributed model training method and device, and the method comprises the steps: enabling each first participant to distribute a sample data block to different training processes, and carrying out the forward propagation based on a client model, forward in...

Full description

Saved in:
Bibliographic Details
Main Author FENG SHIPENG
Format Patent
LanguageChinese
English
Published 19.07.2024
Subjects
CALCULATING
COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
COMPUTING
COUNTING
PHYSICS
Online AccessGet full text

Cover

More Information
Summary:The embodiment of the invention provides a distributed model training method and device, and the method comprises the steps: enabling each first participant to distribute a sample data block to different training processes, and carrying out the forward propagation based on a client model, forward intermediate results obtained by forward propagation of different training processes are sent to the second participant, the second participant allocates the forward intermediate results to the corresponding training processes to perform forward propagation and reverse propagation based on the server model, and reverse intermediate results obtained by reverse propagation are sent to the first participants; carrying out gradient data synchronization on gradient data obtained in back propagation and other training processes and updating server model parameters, and distributing a back intermediate result to the corresponding training process by each first participant to carry out back propagation based on a client mode
Bibliography:Application Number: CN202410607177