Fast Conditional Network Compression Using Bayesian HyperNetworks

• Published in: Machine Learning and Knowledge Discovery in Databases. Research Track Vol. 12977; pp. 330 - 345
• Main Authors: Nguyen, Phuoc, Tran, Truyen, Le, Ky, Gupta, Sunil, Rana, Santu, Nguyen, Dang, Nguyen, Trong, Ryan, Shannon, Venkatesh, Svetha
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2021; Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Bayesian compression; Bayesian deep learning; Hyper networks; Meta-learning; Network compression

We introduce a conditional compression problem and propose a fast framework for tackling it. The problem is how to quickly compress a pretrained large neural network into optimal smaller networks given target contexts, e.g., a context involving only a subset of classes or a context where only limite...