Model Compression for Deep Reinforcement Learning Through Mutual Information

• Published in: Advances in Artificial Intelligence – IBERAMIA 2022 pp. 196 - 207
• Main Authors: García-Ramírez, Jesús, Morales, Eduardo F., Escalante, Hugo Jair
• Format: Book Chapter
• Language: English
• Published: Cham Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Deep reinforcement learning; Model compression; Mutual information
• ISBN: 3031224183; 9783031224188
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-031-22419-5_17

One of the most important limitation of deep learning and deep reinforcement learning, is the number of parameters in their models (dozens to hundreds of millions). Different model compression techniques, such as policy distillation, have been proposed to alleviate this limitation. However, they need a high number of instances to obtain acceptable performance and the use of the source model. In this work, we propose a model compression method based on the comparison of mutual information between the distribution layers of the network. This method automatically determines how much the model should be reduced, and the number of instances required to obtain acceptable performance is considerably lower than the state-of-the-art solutions (19M). It also requires lower resources because only the last two layers of the network are fine-tuned.