Bayesian Optimization with Clustering and Rollback for CNN Auto Pruning

Pruning is an effective technique for convolutional neural networks (CNNs) model compression, but it is difficult to find the optimal pruning policy due to the large design space. To improve the usability of pruning, many auto pruning methods have been developed. Recently, Bayesian optimization (BO)...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Fan, Hanwei, Mu, Jiandong, Zhang, Wei
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 25.07.2022
Subjects
Accuracy
Algorithms
Artificial neural networks
Bayesian analysis
Clustering
Machine learning
Model accuracy
Optimization
Pruning
Online AccessGet full text

Cover

More Information
Summary:Pruning is an effective technique for convolutional neural networks (CNNs) model compression, but it is difficult to find the optimal pruning policy due to the large design space. To improve the usability of pruning, many auto pruning methods have been developed. Recently, Bayesian optimization (BO) has been considered to be a competitive algorithm for auto pruning due to its solid theoretical foundation and high sampling efficiency. However, BO suffers from the curse of dimensionality. The performance of BO deteriorates when pruning deep CNNs, since the dimension of the design spaces increase. We propose a novel clustering algorithm that reduces the dimension of the design space to speed up the searching process. Subsequently, a rollback algorithm is proposed to recover the high-dimensional design space so that higher pruning accuracy can be obtained. We validate our proposed method on ResNet, MobileNetV1, and MobileNetV2 models. Experiments show that the proposed method significantly improves the convergence rate of BO when pruning deep CNNs with no increase in running time. The source code is available at https://github.com/fanhanwei/BOCR.
ISSN:2331-8422