HARDWARE NODE WITH MATRIX-VECTOR MULTIPLY TILES FOR NEURAL NETWORK PROCESSING

• Main Authors: CHUNG, Eric S, FOWERS, Jeremy
• Format: Patent
• Language: English; French; German
• Published: 19.07.2023
• Subjects: BASIC ELECTRONIC CIRCUITRY; CALCULATING; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS; COMPUTING; COUNTING; ELECTRIC DIGITAL DATA PROCESSING; ELECTRICITY; PHYSICS; PULSE TECHNIQUE

Systems and methods for neural network processing are provided. A method in a system comprising a plurality of nodes interconnected via a network, where each node includes a plurality of on-chip memory blocks and a plurality of compute units, is provided. The method includes upon service activation receiving an N by M matrix of coefficients corresponding to the neural network model. The method includes loading the coefficients corresponding to the neural network model into the plurality of the on-chip memory blocks for processing by the plurality of compute units. The method includes regardless of a utilization of the plurality of the on-chip memory blocks as part of an evaluation of the neural network model, maintaining the coefficients corresponding to the neural network model in the plurality of the on-chip memory blocks until the service is interrupted or the neural network model is modified or replaced.