METHOD, DEVICE, CHIP AND SYSTEM FOR TRAINING NEURAL NETWORK MODEL

• Main Authors: TU, Dandan, ZHANG, Changzheng, BAI, Xiaolong
• Format: Patent
• Language: Chinese; English; French
• Published: 07.06.2018
• Subjects: CALCULATING; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS; COMPUTING; COUNTING; PHYSICS

Provided are a method, device, chip and system for training a neural network model capable of reducing a delay in training a model parameter. Each training cycle comprises K iterations. The i th iteration within each training cycle for one of N workers is executed in parallel by the respective workers. A model parameter of the i+1th iteration is calculated according to a local gradient and a model parameter of the i th iteration. If i is less than K, a local gradient of the i+1th iteration is calculated according to the model parameter and sample data of the i+1th iteration. The workers pull, from a server module, a global gradient of the r th iteration and/or push, to the server module, a local gradient of the f th iteration. The present invention enables time windows of a calculation process and a communication process to overlap, thereby reducing a delay in training a model parameter. L'invention concerne un procédé, un dispositif, une puce et un système d'apprentissage d'un modèle de réseau neuronal capab