ASYNCHRONOUS NEURAL NETWORK TRAINING

A neural network training apparatus is described which has a network of worker nodes each having a memory storing a subgraph of a neural network to be trained. The apparatus has a control node connected to the network of worker nodes. The control node is configured to send training data instances in...

Full description

Saved in:
Bibliographic Details
Main Authors TARLOW, Daniel Stefan, JOHNSON, Matthew Alastair, RIECHERT, Maik, WEBSTER, Samuel Alexander, TOMIOKA, Ryota, GAUNT, Alexander Lloyd, VYTINIOTIS, Dimitrios
Format Patent
LanguageEnglish
Published 22.11.2018
Subjects
CALCULATING
COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
COMPUTING
COUNTING
PHYSICS
Online AccessGet full text

Cover

More Information
Summary:A neural network training apparatus is described which has a network of worker nodes each having a memory storing a subgraph of a neural network to be trained. The apparatus has a control node connected to the network of worker nodes. The control node is configured to send training data instances into the network to trigger parallelized message passing operations which implement a training algorithm which trains the neural network. At least some of the message passing operations asynchronously update parameters of individual subgraphs of the neural network at the individual worker nodes.
Bibliography:Application Number: US201715599058