Stage-Wise Magnitude-Based Pruning for Recurrent Neural Networks

• Published in: IEEE transaction on neural networks and learning systems Vol. 35; no. 2; pp. 1666 - 1680
• Main Authors: Li, Guiying, Yang, Peng, Qian, Chao, Hong, Richang, Tang, Ke
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.02.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Artificial neural networks; Computational modeling; Deep neural network (DNN) compression; Hardware; language models; machine translation; Natural language processing; Neural networks; Neurons; optimization; Pruning; Recurrent neural networks; recurrent neural networks (RNNs); Sparse matrices; Task analysis; Training

A recurrent neural network (RNN) has shown powerful performance in tackling various natural language processing (NLP) tasks, resulting in numerous powerful models containing both RNN neurons and feedforward neurons. On the other hand, the deep structure of RNN has heavily restricted its implementati...