EXPLOITING ACTIVATION SPARSITY IN DEEP NEURAL NETWORKS

• Main Authors: ANSARI ANSAR, GOLDFARB MICHAEL, LOTT CHRISTOPHER, LAMB AARON, HILL REXFORD
• Format: Patent
• Language: Chinese; English
• Published: 30.04.2021
• Subjects: CALCULATING; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS; COMPUTING; COUNTING; PHYSICS

A method of exploiting activation sparsity in deep neural networks is described. The method includes retrieving an activation tensor and a weight tensor, wherein the activation tensor is a sparse activation tensor. The method also includes generating a compressed activation tensor comprising non-zero activations of the activation tensor, wherein the compressed activation tensor has fewer columns than the activation tensor. The method further includes processing the compressed activation tensor and the weight tensor to generate an output tensor. 描述了一种在深度神经网络中利用激活稀疏性的方法。该方法包括检索激活张量和权重张量，其中该激活张量是稀疏激活张量。该方法还包括生成包含该激活张量的非零激活的经压缩激活张量，其中该经压缩激活张量具有比该激活张量少的列。该方法进一步包括对该经压缩激活张量和该权重张量进行处理以生成输出张量。