Distributed training system, neural network model training method, equipment and medium

• Main Authors: GUO ZHENHUA, LI RENGANG, QIU ZHIYONG, YAN RUIDONG
• Format: Patent
• Language: Chinese; English
• Published: 25.08.2023
• Subjects: CALCULATING; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS; COMPUTING; COUNTING; PHYSICS

The invention discloses a distributed training system, a neural network model training method, equipment and a medium, and relates to the technical field of computers, the system comprises a controller and a plurality of computing nodes; the controller is used for acquiring a target neural network model and a training data set, and sending a corresponding node training model and a node training data set to the plurality of computing nodes; the target neural network model is a neural network model obtained by removing a first number of links with the lowest link importance degree between the ith neural network layer and the (i + 1) th neural network layer, and removing a second number of neurons with the lowest neuron importance degree and the corresponding links from the initial neural network model; and the computing node is used for computing local information based on the received node training data set and the received current model parameters of the node training model, updating the model parameters of t