Training a neural network

• Main Authors: Elia Condorelli, Muhammad Asad, Cagatay Dikici
• Format: Patent
• Language: English
• Published: 03.04.2024
• Subjects: CALCULATING; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS; COMPUTING; COUNTING; PHYSICS

A method of training a neural network which combines a set of coefficients with input data values. The method begins by training a test implementation of the neural network. Sparsity is applied based on a threshold which is set using an empirical (e.g. ascending or descending) sorting (of values) of neural network coefficients (weights/multipliers)) and using this, and a sparsity parameter. The test neural network is then operated on training input data using the coefficients to produce training output data. The sparsity parameter is updated based on the accuracy of the test neural network, and a runtime implementation of the neural network is configured using the updated sparsity parameter. Training the test neural network may be performed iteratively and may include updating the coefficients based on the accuracy of the test neural network. Applying sparsity to coefficients may comprise setting a coefficient or group of coefficients to zero. The sparsity parameter may be updated based on an optimisation technique which balances the level of sparsity against accuracy, such as a cross-entropy loss equation.