AutoPruner: An end-to-end trainable filter pruning method for efficient deep model inference
•Filter selection and model fine-tuning are integrated into a single end-to-end trainable framework.•Adaptive compression ratio and multi-layer compression.•Good generalization ability. Channel pruning is an important method to speed up CNN model’s inference. Previous filter pruning algorithms regar...
Saved in:
| Published in | Pattern recognition Vol. 107; p. 107461 |
|---|---|
| Main Authors | , |
| Format | Journal Article |
| Language | English |
| Published |
Elsevier Ltd
01.11.2020
|
| Subjects | |
| Online Access | Get full text |
Cover
Loading…
| Abstract | •Filter selection and model fine-tuning are integrated into a single end-to-end trainable framework.•Adaptive compression ratio and multi-layer compression.•Good generalization ability.
Channel pruning is an important method to speed up CNN model’s inference. Previous filter pruning algorithms regard importance evaluation and model fine-tuning as two independent steps. This paper argues that combining them into a single end-to-end trainable system will lead to better results. We propose an efficient channel selection layer, namely AutoPruner, to find less important filters automatically in a joint training manner. Our AutoPruner takes previous activation responses as an input and generates a true binary index code for pruning. Hence, all the filters corresponding to zero index values can be removed safely after training. By gradually erasing several unimportant filters, we can prevent an excessive drop in model accuracy. Compared with previous state-of-the-art pruning algorithms (including training from scratch), AutoPruner achieves significantly better performance. Furthermore, ablation experiments show that the proposed novel mini-batch pooling and binarization operations are vital for the success of model pruning. |
|---|---|
| AbstractList | •Filter selection and model fine-tuning are integrated into a single end-to-end trainable framework.•Adaptive compression ratio and multi-layer compression.•Good generalization ability.
Channel pruning is an important method to speed up CNN model’s inference. Previous filter pruning algorithms regard importance evaluation and model fine-tuning as two independent steps. This paper argues that combining them into a single end-to-end trainable system will lead to better results. We propose an efficient channel selection layer, namely AutoPruner, to find less important filters automatically in a joint training manner. Our AutoPruner takes previous activation responses as an input and generates a true binary index code for pruning. Hence, all the filters corresponding to zero index values can be removed safely after training. By gradually erasing several unimportant filters, we can prevent an excessive drop in model accuracy. Compared with previous state-of-the-art pruning algorithms (including training from scratch), AutoPruner achieves significantly better performance. Furthermore, ablation experiments show that the proposed novel mini-batch pooling and binarization operations are vital for the success of model pruning. |
| ArticleNumber | 107461 |
| Author | Wu, Jianxin Luo, Jian-Hao |
| Author_xml | – sequence: 1 givenname: Jian-Hao orcidid: 0000-0001-8105-805X surname: Luo fullname: Luo, Jian-Hao – sequence: 2 givenname: Jianxin surname: Wu fullname: Wu, Jianxin email: wujx2001@nju.edu.cn |
| BookMark | eNqFkF1LwzAUhoNMcJv-Ay_yBzqTtE2aXQhj-AUDvdA7IbTJyczokpJmgv_elnrlhV69cM55XjjPAs188IDQNSUrSii_Oay6OumwXzHCxpEoOD1Dc1qJPCtpwWZoTkhOs5yR_AIt-v5ACBXDYo7eN6cUXuLJQ1zjjcfgTZZCNgROsXa-blrA1rUJIu6GM-f3-AjpIxhsQ8RgrdMOfMIGoMPHYKDFzluI4DVconNbtz1c_eQSvd3fvW4fs93zw9N2s8t0LljKGiKNkbXgUouC0ELLSlpSUMugKqUgxDIupCy1FII3ecV5xbRtaCFKrrko8yUqpl4dQ99HsKqL7ljHL0WJGg2pg5oMqdGQmgwN2PoXpl2qkwt-fL39D76dYBge-3QQVT-K0GBcBJ2UCe7vgm8qFoWz |
| CitedBy_id | crossref_primary_10_1007_s10489_022_03293_x crossref_primary_10_1109_TNNLS_2022_3184730 crossref_primary_10_1109_TNNLS_2024_3353763 crossref_primary_10_1016_j_patcog_2021_108056 crossref_primary_10_1007_s13042_023_02076_1 crossref_primary_10_1016_j_patcog_2022_109025 crossref_primary_10_1002_cpe_7143 crossref_primary_10_1016_j_jag_2023_103244 crossref_primary_10_3390_math11234849 crossref_primary_10_1016_j_patcog_2021_107923 crossref_primary_10_1016_j_neucom_2021_07_045 crossref_primary_10_1016_j_neucom_2024_127698 crossref_primary_10_1155_2021_9971669 crossref_primary_10_1371_journal_pone_0319859 crossref_primary_10_1016_j_neucom_2024_128461 crossref_primary_10_1109_TCSVT_2022_3156588 crossref_primary_10_1016_j_patcog_2023_109508 crossref_primary_10_1109_TCAD_2023_3276938 crossref_primary_10_1109_TNNLS_2021_3084856 crossref_primary_10_1007_s00521_024_09719_6 crossref_primary_10_1109_TPAMI_2023_3323496 crossref_primary_10_1007_s11554_022_01209_z crossref_primary_10_1016_j_cviu_2024_103942 crossref_primary_10_1109_LCA_2021_3101505 crossref_primary_10_1007_s44267_024_00040_3 crossref_primary_10_1109_TPAMI_2022_3226777 crossref_primary_10_1016_j_patcog_2021_108366 crossref_primary_10_1016_j_neucom_2024_127817 crossref_primary_10_1109_TPAMI_2023_3260903 crossref_primary_10_3390_informatics8040077 crossref_primary_10_1007_s10489_025_06332_5 crossref_primary_10_1109_ACCESS_2022_3182659 crossref_primary_10_1109_ACCESS_2022_3140807 crossref_primary_10_3390_app14041491 crossref_primary_10_1109_TNNLS_2022_3147269 crossref_primary_10_1109_TPAMI_2023_3259402 crossref_primary_10_1093_jcde_qwad038 crossref_primary_10_1016_j_neunet_2022_05_002 crossref_primary_10_1016_j_neunet_2024_106263 crossref_primary_10_1109_ACCESS_2024_3454329 crossref_primary_10_1109_ACCESS_2025_3551802 crossref_primary_10_1016_j_eswa_2024_126104 crossref_primary_10_1007_s11633_022_1340_5 crossref_primary_10_1016_j_patcog_2024_110671 crossref_primary_10_1016_j_cviu_2024_104247 crossref_primary_10_1109_TIP_2021_3112041 crossref_primary_10_1142_S1469026823500256 crossref_primary_10_1016_j_neunet_2024_106393 crossref_primary_10_1016_j_patcog_2023_109886 crossref_primary_10_1016_j_suscom_2022_100814 crossref_primary_10_1109_TNNLS_2023_3280899 crossref_primary_10_1109_TGRS_2025_3545416 crossref_primary_10_1016_j_neucom_2024_128488 crossref_primary_10_1109_TCSVT_2022_3175762 crossref_primary_10_1109_TCSVT_2024_3488098 crossref_primary_10_1109_TNNLS_2022_3156047 crossref_primary_10_1007_s10489_024_05690_w crossref_primary_10_1007_s11633_022_1353_0 crossref_primary_10_1016_j_patcog_2023_109847 crossref_primary_10_1109_JPROC_2021_3119950 crossref_primary_10_1016_j_neucom_2023_02_004 crossref_primary_10_1016_j_neunet_2021_12_020 crossref_primary_10_1016_j_patcog_2024_110488 crossref_primary_10_3390_rs15123144 crossref_primary_10_1007_s11042_023_15608_2 crossref_primary_10_1016_j_asoc_2022_109856 crossref_primary_10_3390_s22218427 crossref_primary_10_1016_j_neucom_2021_08_158 crossref_primary_10_1007_s10462_022_10141_4 crossref_primary_10_1109_TDSC_2023_3267065 crossref_primary_10_1109_JIOT_2021_3116316 crossref_primary_10_1007_s12293_024_00406_6 crossref_primary_10_1016_j_cviu_2021_103220 crossref_primary_10_1016_j_eswa_2022_118313 crossref_primary_10_1007_s11554_022_01227_x crossref_primary_10_1016_j_dsp_2022_103408 crossref_primary_10_1016_j_ins_2024_120155 crossref_primary_10_1016_j_neunet_2023_01_024 crossref_primary_10_3390_informatics10030072 crossref_primary_10_1109_TCE_2024_3475517 crossref_primary_10_2355_isijinternational_ISIJINT_2023_360 crossref_primary_10_1007_s10489_022_03508_1 crossref_primary_10_3390_app122110952 crossref_primary_10_1007_s10489_022_03774_z crossref_primary_10_3390_s22124331 crossref_primary_10_1016_j_patcog_2022_108729 |
| Cites_doi | 10.1109/TNNLS.2019.2906563 10.1007/s11263-015-0816-y 10.1109/TPAMI.2018.2858232 10.5244/C.28.88 10.1109/TPAMI.2018.2873305 10.1007/s11263-009-0275-4 10.1016/j.patcog.2017.06.032 10.1016/j.patcog.2018.10.029 10.1109/5.726791 10.1109/TPAMI.2015.2502579 |
| ContentType | Journal Article |
| Copyright | 2020 Elsevier Ltd |
| Copyright_xml | – notice: 2020 Elsevier Ltd |
| DBID | AAYXX CITATION |
| DOI | 10.1016/j.patcog.2020.107461 |
| DatabaseName | CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISSN | 1873-5142 |
| ExternalDocumentID | 10_1016_j_patcog_2020_107461 S0031320320302648 |
| GroupedDBID | --K --M -D8 -DT -~X .DC .~1 0R~ 123 1B1 1RT 1~. 1~5 29O 4.4 457 4G. 53G 5VS 7-5 71M 8P~ 9JN AABNK AACTN AAEDT AAEDW AAIAV AAIKJ AAKOC AALRI AAOAW AAQFI AAQXK AAXUO AAYFN ABBOA ABEFU ABFNM ABFRF ABHFT ABJNI ABMAC ABTAH ABXDB ABYKQ ACBEA ACDAQ ACGFO ACGFS ACNNM ACRLP ACZNC ADBBV ADEZE ADJOM ADMUD ADMXK ADTZH AEBSH AECPX AEFWE AEKER AENEX AFKWA AFTJW AGHFR AGUBO AGYEJ AHHHB AHJVU AHZHX AIALX AIEXJ AIKHN AITUG AJBFU AJOXV ALMA_UNASSIGNED_HOLDINGS AMFUW AMRAJ AOUOD ASPBG AVWKF AXJTR AZFZN BJAXD BKOJK BLXMC CS3 DU5 EBS EFJIC EFLBG EJD EO8 EO9 EP2 EP3 F0J F5P FD6 FDB FEDTE FGOYB FIRID FNPLU FYGXN G-Q G8K GBLVA GBOLZ HLZ HVGLF HZ~ H~9 IHE J1W JJJVA KOM KZ1 LG9 LMP LY1 M41 MO0 N9A O-L O9- OAUVE OZT P-8 P-9 P2P PC. Q38 R2- RIG RNS ROL RPZ SBC SDF SDG SDP SDS SES SEW SPC SPCBC SST SSV SSZ T5K TN5 UNMZH VOH WUQ XJE XPP ZMT ZY4 ~G- AATTM AAXKI AAYWO AAYXX ABDPE ABWVN ACRPL ACVFH ADCNI ADNMO AEIPS AEUPX AFJKZ AFPUW AFXIZ AGCQF AGQPQ AGRNS AIGII AIIUN AKBMS AKRWK AKYEP ANKPU APXCP BNPGV CITATION SSH |
| ID | FETCH-LOGICAL-c372t-b09dd9a769c74014c989f041f2e859700f267995c9776b386682cfb14756c6753 |
| IEDL.DBID | .~1 |
| ISSN | 0031-3203 |
| IngestDate | Thu Apr 24 23:07:55 EDT 2025 Tue Jul 01 02:36:31 EDT 2025 Fri Feb 23 02:49:04 EST 2024 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Keywords | CNN acceleration Neural network pruning Model compression |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c372t-b09dd9a769c74014c989f041f2e859700f267995c9776b386682cfb14756c6753 |
| ORCID | 0000-0001-8105-805X |
| ParticipantIDs | crossref_primary_10_1016_j_patcog_2020_107461 crossref_citationtrail_10_1016_j_patcog_2020_107461 elsevier_sciencedirect_doi_10_1016_j_patcog_2020_107461 |
| ProviderPackageCode | CITATION AAYXX |
| PublicationCentury | 2000 |
| PublicationDate | November 2020 2020-11-00 |
| PublicationDateYYYYMMDD | 2020-11-01 |
| PublicationDate_xml | – month: 11 year: 2020 text: November 2020 |
| PublicationDecade | 2020 |
| PublicationTitle | Pattern recognition |
| PublicationYear | 2020 |
| Publisher | Elsevier Ltd |
| Publisher_xml | – name: Elsevier Ltd |
| References | Denil, Shakibi, Dinh, de Freitas (bib0009) 2013 Huang, Wang (bib0005) 2018; 11220 M. Jaderberg, A. Vedaldi, A. Zisserman, Speeding up convolutional neural networks with low rank expansions, arXiv Liu, Li, Shen, Huang, Yan, Zhang (bib0017) 2017 He, Zhang, Sun (bib0016) 2017 Russakovsky, Deng, Su, Krause, Satheesh, Ma, Huang, Karpathy, Khosla, Bernstein, Berg, Li (bib0019) 2015; 115 Liu, Anguelov, Erhan, Szegedy, Reed, Fu, Berg (bib0052) 2016; 9905 Yu, Yang, Xu, Yang, Huang (bib0049) 2019 Lin, Maire, Belongie, Hays, Perona, Ramanan, Dollár, Zitnick (bib0051) 2014; 8693 He, Lin, Liu, Wang, Li, Han (bib0027) 2018; 11211 Lin, Ji, Guo, Li (bib0037) 2016 He, Zhang, Ren, Sun (bib0040) 2016 Molchanov, Mallya, Tyree, Frosio, Kautz (bib0045) 2019 Aghasi, Abdi, Nguyen, Romberg (bib0022) 2017 Hu, Shen, Sun (bib0054) 2018 Lin, Ji, Li, Deng, Li (bib0030) 2020; 31 Liu, Anguelov, Erhan, Szegedy, Reed, Fu, Berg (bib0038) 2016; 9908 Lin, Ji, Chen, Tao, Luo (bib0036) 2019; 41 Xiao, Jin, Yang, Yang, Sun, Chang (bib0013) 2017; 72 Molchanov, Tyree, Karras, Aila, Kautz (bib0042) 2017 (2015). Wen, He, Rajbhandari, Wang, Liu, Hu, Chen, Li (bib0025) 2018 Alvarez, Salzmann (bib0021) 2017 Gao, Zhao, Dudziak, Mullins, Xu (bib0035) 2019 Chen, Wilson, Tyree, Weinberger, Chen (bib0006) 2015 Dong, Huang, Yang, Yan (bib0031) 2017 Lebedev, Lempitsky (bib0007) 2016 (2019). (2017). Li, Kadav, Durdanovic, Samet, Graf (bib0023) 2017 Zhang, Zou, He, Sun (bib0048) 2016; 38 2014. Han, Pool, Tran, Dally (bib0004) 2015 Singh, Verma, Rai, Namboodiri (bib0028) 2019 Lin, Ji, Li, Wu, Huang, Zhang (bib0033) 2018 Wen, Wu, Wang, Chen, Li (bib0014) 2016 G.B. Hacene, C. Lassance, V. Gripon, M. Courbariaux, Y. Bengio, Attention based pruning for shift networks, arXiv Xu, Yang, Zhang, Liu (bib0011) 2019; 88 Lin, Ji, Yan, Zhang, Cao, Ye, Huang, Doermann (bib0046) 2019 Yu, Li, Chen, Lai, Morariu, Han, Gao, Lin, Davis (bib0026) 2018 Luo, Wu, Lin (bib0002) 2017 Lin, Rao, Lu, Zhou (bib0015) 2017 Krizhevsky (bib0044) 2009 Louizos, Ullrich, Welling (bib0020) 2017 H. Wang, Q. Zhang, Y. Wang, H. Hu, Structured probabilistic pruning for convolutional neural network acceleration, arXiv LeCun, Bottou, Bengio, Haffner (bib0043) 1998; 86 Everingham, Van Gool, Williams, Winn, Zisserman (bib0050) 2010; 88 Ding, Chen, Huo (bib0012) 2019; 96 Wah, Branson, Welinder, Perona, Belongie (bib0018) 2011 Sandler, Howard, Zhu, Zhmoginov, Chen (bib0041) 2018 Denton, Zaremba, Bruna, LeCun, Fergus (bib0008) 2014 Liu, Sun, Zhou, Huang, Darrell (bib0053) 2019 Han, Mao, Dally (bib0024) 2016 Luo, Zhang, Zhou, Xie, Wu, Lin (bib0003) 2019; 41 Wu, Leng, Wang, Hu, Cheng (bib0010) 2016 Wang, Zhang, Wang, Yu, Hu (bib0029) 2019 G. Hinton, O. Vinyals, J. Dean, Distilling the knowledge in a neural network, arXiv Simonyan, Zisserman (bib0001) 2015 Lin (10.1016/j.patcog.2020.107461_bib0037) 2016 He (10.1016/j.patcog.2020.107461_bib0040) 2016 Luo (10.1016/j.patcog.2020.107461_bib0002) 2017 Huang (10.1016/j.patcog.2020.107461_bib0005) 2018; 11220 10.1016/j.patcog.2020.107461_bib0047 Lin (10.1016/j.patcog.2020.107461_bib0030) 2020; 31 Wang (10.1016/j.patcog.2020.107461_bib0029) 2019 Liu (10.1016/j.patcog.2020.107461_bib0038) 2016; 9908 Lin (10.1016/j.patcog.2020.107461_bib0036) 2019; 41 Denil (10.1016/j.patcog.2020.107461_bib0009) 2013 Ding (10.1016/j.patcog.2020.107461_bib0012) 2019; 96 Yu (10.1016/j.patcog.2020.107461_bib0049) 2019 Lin (10.1016/j.patcog.2020.107461_bib0033) 2018 Denton (10.1016/j.patcog.2020.107461_bib0008) 2014 Russakovsky (10.1016/j.patcog.2020.107461_bib0019) 2015; 115 Krizhevsky (10.1016/j.patcog.2020.107461_bib0044) 2009 Louizos (10.1016/j.patcog.2020.107461_bib0020) 2017 Gao (10.1016/j.patcog.2020.107461_bib0035) 2019 Sandler (10.1016/j.patcog.2020.107461_bib0041) 2018 Hu (10.1016/j.patcog.2020.107461_bib0054) 2018 Liu (10.1016/j.patcog.2020.107461_bib0017) 2017 Alvarez (10.1016/j.patcog.2020.107461_bib0021) 2017 Li (10.1016/j.patcog.2020.107461_bib0023) 2017 Wen (10.1016/j.patcog.2020.107461_bib0025) 2018 Luo (10.1016/j.patcog.2020.107461_bib0003) 2019; 41 Molchanov (10.1016/j.patcog.2020.107461_bib0042) 2017 Zhang (10.1016/j.patcog.2020.107461_bib0048) 2016; 38 He (10.1016/j.patcog.2020.107461_bib0027) 2018; 11211 Singh (10.1016/j.patcog.2020.107461_bib0028) 2019 Xu (10.1016/j.patcog.2020.107461_bib0011) 2019; 88 Yu (10.1016/j.patcog.2020.107461_bib0026) 2018 Liu (10.1016/j.patcog.2020.107461_bib0053) 2019 Han (10.1016/j.patcog.2020.107461_bib0004) 2015 Dong (10.1016/j.patcog.2020.107461_bib0031) 2017 LeCun (10.1016/j.patcog.2020.107461_bib0043) 1998; 86 Han (10.1016/j.patcog.2020.107461_bib0024) 2016 Wen (10.1016/j.patcog.2020.107461_bib0014) 2016 Lin (10.1016/j.patcog.2020.107461_bib0046) 2019 Chen (10.1016/j.patcog.2020.107461_bib0006) 2015 Xiao (10.1016/j.patcog.2020.107461_bib0013) 2017; 72 10.1016/j.patcog.2020.107461_bib0034 Lebedev (10.1016/j.patcog.2020.107461_bib0007) 2016 Lin (10.1016/j.patcog.2020.107461_bib0051) 2014; 8693 Molchanov (10.1016/j.patcog.2020.107461_bib0045) 2019 10.1016/j.patcog.2020.107461_bib0039 Lin (10.1016/j.patcog.2020.107461_bib0015) 2017 Aghasi (10.1016/j.patcog.2020.107461_bib0022) 2017 10.1016/j.patcog.2020.107461_bib0032 Wah (10.1016/j.patcog.2020.107461_bib0018) 2011 Everingham (10.1016/j.patcog.2020.107461_bib0050) 2010; 88 Simonyan (10.1016/j.patcog.2020.107461_bib0001) 2015 Wu (10.1016/j.patcog.2020.107461_bib0010) 2016 He (10.1016/j.patcog.2020.107461_bib0016) 2017 Liu (10.1016/j.patcog.2020.107461_bib0052) 2016; 9905 |
| References_xml | – start-page: 1135 year: 2015 end-page: 1143 ident: bib0004 article-title: Learning both weights and connections for efficient neural network publication-title: Proceedings of Advances in Neural Information Processing Systems (NeurIPS) – start-page: 3180 year: 2017 end-page: 3189 ident: bib0022 article-title: Net-Trim: convex pruning of deep neural networks with performance guarantee publication-title: Proceedings of Advances in Neural Information Processing Systems (NeurIPS) – reference: H. Wang, Q. Zhang, Y. Wang, H. Hu, Structured probabilistic pruning for convolutional neural network acceleration, arXiv: – volume: 9905 start-page: 21 year: 2016 end-page: 37 ident: bib0052 article-title: SSD: single shot multibox detector publication-title: Proceeding of European Conference on Computer Vision (ECCV) – start-page: 1753 year: 2016 end-page: 1759 ident: bib0037 article-title: Towards convolutional neural networks compression via global error reconstruction publication-title: Proceedings of International Joint Conference on Artificial Intelligence (IJCAI) – year: 2018 ident: bib0025 article-title: Learning intrinsic sparse structures within long short-term memory publication-title: Proceedings of International Conference on Learning Representations (ICLR) – year: 2017 ident: bib0042 article-title: Pruning convolutional neural networks for resource efficient inference publication-title: Proceedings of International Conference on Learning Representations (ICLR) – start-page: 2755 year: 2017 end-page: 2763 ident: bib0017 article-title: Learning efficient convolutional networks through network slimming publication-title: Proceedings of International Conference on Computer Vision (ICCV) – volume: 8693 start-page: 740 year: 2014 end-page: 755 ident: bib0051 article-title: Microsoft COCO: common objects in context publication-title: European Conference on Computer Vision (ECCV) – reference: (2015). – volume: 11211 start-page: 815 year: 2018 end-page: 832 ident: bib0027 article-title: AMC: AutoML for model compression and acceleration on mobile devices publication-title: Proceeding of European Conference on Computer Vision (ECCV) – start-page: 7132 year: 2018 end-page: 7141 ident: bib0054 article-title: Squeeze-and-excitation networks publication-title: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) – year: 2019 ident: bib0053 article-title: Rethinking the value of network pruning publication-title: Proceedings of International Conference on Learning Representations (ICLR) – start-page: 5058 year: 2017 end-page: 5066 ident: bib0002 article-title: ThiNet: a filter level pruning method for deep neural network compression publication-title: Proceedings of International Conference on Computer Vision (ICCV) – start-page: 1389 year: 2017 end-page: 1397 ident: bib0016 article-title: Channel pruning for accelerating very deep neural networks publication-title: Proceedings of International Conference on Computer Vision (ICCV) – volume: 86 start-page: 2278 year: 1998 end-page: 2324 ident: bib0043 article-title: Gradient-based learning applied to document recognition publication-title: Proc. IEEE – year: 2018 ident: bib0026 article-title: NISP: pruning networks using neuron importance score propagation publication-title: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) – volume: 9908 start-page: 525 year: 2016 end-page: 542 ident: bib0038 article-title: XNOR-Net: ImageNet classification using binary convolutional neural networks publication-title: European Conference on Computer Vision (ECCV) – volume: 38 start-page: 1943 year: 2016 end-page: 1955 ident: bib0048 article-title: Accelerating very deep convolutional networks for classification and detection publication-title: IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) – reference: G.B. Hacene, C. Lassance, V. Gripon, M. Courbariaux, Y. Bengio, Attention based pruning for shift networks, arXiv: – start-page: 2148 year: 2013 end-page: 2156 ident: bib0009 article-title: Predicting parameters in deep learning publication-title: Proceedings of Advances in Neural Information Processing Systems (NeurIPS) – year: 2015 ident: bib0001 article-title: Very deep convolutional networks for large-scale image recognition publication-title: Proceedings of International Conference on Learning Representations (ICLR) – volume: 31 start-page: 574 year: 2020 end-page: 588 ident: bib0030 article-title: Toward compact ConvNets via structure-sparsity regularized filter pruning publication-title: IEEE Trans. Neural Netw. Learn. Syst. (TNNLS) – year: 2017 ident: bib0023 article-title: Pruning filters for efficient ConvNets publication-title: Proceedings of International Conference on Learning Representations (ICLR) – reference: (2017). – start-page: 3460 year: 2019 end-page: 3466 ident: bib0028 article-title: Play and prune: adaptive filter pruning for deep model compression publication-title: Proceedings of International Joint Conference on Artificial Intelligence (IJCAI) – start-page: 2790 year: 2019 end-page: 2799 ident: bib0046 article-title: Towards optimal structured CNN pruning via generative adversarial learning publication-title: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) – year: 2019 ident: bib0049 article-title: Slimmable neural networks publication-title: Proceedings of International Conference on Learning Representations (ICLR) – volume: 115 start-page: 211 year: 2015 end-page: 252 ident: bib0019 article-title: ImageNet large scale visual recognition challenge publication-title: Int. J. Comput. Vis. (IJCV) – start-page: 11264 year: 2019 end-page: 11272 ident: bib0045 article-title: Importance estimation for neural network pruning publication-title: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) – year: 2019 ident: bib0035 article-title: Dynamic channel pruning: feature boosting and suppression publication-title: Proceedings of International Conference on Learning Representations (ICLR) – start-page: 2425 year: 2018 end-page: 2432 ident: bib0033 article-title: Accelerating convolutional networks via global & dynamic filter pruning publication-title: Proceedings of International Joint Conference on Artificial Intelligence (IJCAI) – start-page: 2285 year: 2015 end-page: 2294 ident: bib0006 article-title: Compressing neural networks with the hashing trick publication-title: Proceedings of International Conference on Machine Learning (ICML) – reference: (2019). – start-page: 1269 year: 2014 end-page: 1277 ident: bib0008 article-title: Exploiting linear structure within convolutional networks for efficient evaluation publication-title: Proceedings of Advances in Neural Information Processing Systems (NeurIPS) – start-page: 856 year: 2017 end-page: 867 ident: bib0021 article-title: Compression-aware training of deep networks publication-title: Proceedings of Advances in Neural Information Processing Systems (NeurIPS) – start-page: 2554 year: 2016 end-page: 2564 ident: bib0007 article-title: Fast convnets using group-wise brain damage publication-title: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) – year: 2011 ident: bib0018 article-title: The Caltech-UCSD Birds-200–2011 Dataset publication-title: Technical Report – volume: 11220 start-page: 317 year: 2018 end-page: 334 ident: bib0005 article-title: Data-driven sparse structure selection for deep neural networks publication-title: Proceeding of European Conference on Computer Vision (ECCV) – volume: 96 start-page: 1 year: 2019 end-page: 14 ident: bib0012 article-title: Compressing CNN-DBLSTM models for OCR with teacher-student learning and Tucker decomposition publication-title: Pattern Recognit. (PR) – start-page: 5840 year: 2017 end-page: 5848 ident: bib0031 article-title: More is less: a more complicated network with less inference complexity publication-title: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) – reference: M. Jaderberg, A. Vedaldi, A. Zisserman, Speeding up convolutional neural networks with low rank expansions, arXiv: – start-page: 770 year: 2016 end-page: 778 ident: bib0040 article-title: Deep residual learning for image recognition publication-title: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) – start-page: 2074 year: 2016 end-page: 2082 ident: bib0014 article-title: Learning structured sparsity in deep neural networks publication-title: Proceedings of Advances in Neural Information Processing Systems (NeurIPS) – volume: 88 start-page: 272 year: 2019 end-page: 284 ident: bib0011 article-title: LightweightNet: toward fast and lightweight convolutional neural networks via architecture distillation publication-title: Pattern Recognit. (PR) – start-page: 3290 year: 2017 end-page: 3300 ident: bib0020 article-title: Bayesian compression for deep learning publication-title: Proceedings of Advances in Neural Information Processing Systems (NeurIPS) – start-page: 1 year: 2019 end-page: 8 ident: bib0029 article-title: Structured pruning for efficient ConvNets via incremental regularization publication-title: Proceedings of International Joint Conference on Neural Networks (IJCNN) – reference: G. Hinton, O. Vinyals, J. Dean, Distilling the knowledge in a neural network, arXiv: – start-page: 4820 year: 2016 end-page: 4828 ident: bib0010 article-title: Quantized convolutional neural networks for mobile devices publication-title: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) – volume: 72 start-page: 72 year: 2017 end-page: 81 ident: bib0013 article-title: Building fast and compact convolutional neural networks for offline handwritten chinese character recognition publication-title: Pattern Recognit. (PR) – start-page: 2178 year: 2017 end-page: 2188 ident: bib0015 article-title: Runtime neural pruning publication-title: Proceedings of Advances in Neural Information Processing Systems (NeurIPS) – reference: 2014. – volume: 88 start-page: 303 year: 2010 end-page: 338 ident: bib0050 article-title: The PASCAL visual object classes (VOC) challenge publication-title: Int. J. Comput. Vis. (IJCV) – volume: 41 start-page: 2889 year: 2019 end-page: 2905 ident: bib0036 article-title: Holistic CNN compression via low-rank decomposition with knowledge transfer publication-title: IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) – volume: 41 start-page: 2525 year: 2019 end-page: 2538 ident: bib0003 article-title: ThiNet: pruning CNN filters for a thinner net publication-title: IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) – year: 2009 ident: bib0044 publication-title: Learning multiple layers of features from tiny images – year: 2016 ident: bib0024 article-title: Deep compression: compressing deep neural networks with pruning, trained quantization and Huffman coding publication-title: Proceedings of International Conference on Learning Representations (ICLR) – start-page: 4510 year: 2018 end-page: 4520 ident: bib0041 article-title: MobileNetV2: inverted residuals and linear bottlenecks publication-title: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) – ident: 10.1016/j.patcog.2020.107461_bib0032 – start-page: 3460 year: 2019 ident: 10.1016/j.patcog.2020.107461_bib0028 article-title: Play and prune: adaptive filter pruning for deep model compression – start-page: 1 year: 2019 ident: 10.1016/j.patcog.2020.107461_bib0029 article-title: Structured pruning for efficient ConvNets via incremental regularization – volume: 96 start-page: 1 year: 2019 ident: 10.1016/j.patcog.2020.107461_bib0012 article-title: Compressing CNN-DBLSTM models for OCR with teacher-student learning and Tucker decomposition publication-title: Pattern Recognit. (PR) – start-page: 7132 year: 2018 ident: 10.1016/j.patcog.2020.107461_bib0054 article-title: Squeeze-and-excitation networks – volume: 11211 start-page: 815 year: 2018 ident: 10.1016/j.patcog.2020.107461_bib0027 article-title: AMC: AutoML for model compression and acceleration on mobile devices – year: 2009 ident: 10.1016/j.patcog.2020.107461_bib0044 – ident: 10.1016/j.patcog.2020.107461_bib0039 – year: 2018 ident: 10.1016/j.patcog.2020.107461_bib0026 article-title: NISP: pruning networks using neuron importance score propagation – volume: 8693 start-page: 740 year: 2014 ident: 10.1016/j.patcog.2020.107461_bib0051 article-title: Microsoft COCO: common objects in context – start-page: 2790 year: 2019 ident: 10.1016/j.patcog.2020.107461_bib0046 article-title: Towards optimal structured CNN pruning via generative adversarial learning – start-page: 1135 year: 2015 ident: 10.1016/j.patcog.2020.107461_bib0004 article-title: Learning both weights and connections for efficient neural network – start-page: 2178 year: 2017 ident: 10.1016/j.patcog.2020.107461_bib0015 article-title: Runtime neural pruning – start-page: 2554 year: 2016 ident: 10.1016/j.patcog.2020.107461_bib0007 article-title: Fast convnets using group-wise brain damage – volume: 31 start-page: 574 issue: 2 year: 2020 ident: 10.1016/j.patcog.2020.107461_bib0030 article-title: Toward compact ConvNets via structure-sparsity regularized filter pruning publication-title: IEEE Trans. Neural Netw. Learn. Syst. (TNNLS) doi: 10.1109/TNNLS.2019.2906563 – start-page: 11264 year: 2019 ident: 10.1016/j.patcog.2020.107461_bib0045 article-title: Importance estimation for neural network pruning – volume: 115 start-page: 211 issue: 3 year: 2015 ident: 10.1016/j.patcog.2020.107461_bib0019 article-title: ImageNet large scale visual recognition challenge publication-title: Int. J. Comput. Vis. (IJCV) doi: 10.1007/s11263-015-0816-y – volume: 41 start-page: 2525 issue: 10 year: 2019 ident: 10.1016/j.patcog.2020.107461_bib0003 article-title: ThiNet: pruning CNN filters for a thinner net publication-title: IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) doi: 10.1109/TPAMI.2018.2858232 – start-page: 5840 year: 2017 ident: 10.1016/j.patcog.2020.107461_bib0031 article-title: More is less: a more complicated network with less inference complexity – ident: 10.1016/j.patcog.2020.107461_bib0047 doi: 10.5244/C.28.88 – start-page: 1753 year: 2016 ident: 10.1016/j.patcog.2020.107461_bib0037 article-title: Towards convolutional neural networks compression via global error reconstruction – start-page: 3180 year: 2017 ident: 10.1016/j.patcog.2020.107461_bib0022 article-title: Net-Trim: convex pruning of deep neural networks with performance guarantee – start-page: 770 year: 2016 ident: 10.1016/j.patcog.2020.107461_bib0040 article-title: Deep residual learning for image recognition – year: 2017 ident: 10.1016/j.patcog.2020.107461_bib0023 article-title: Pruning filters for efficient ConvNets – start-page: 2285 year: 2015 ident: 10.1016/j.patcog.2020.107461_bib0006 article-title: Compressing neural networks with the hashing trick – volume: 9905 start-page: 21 year: 2016 ident: 10.1016/j.patcog.2020.107461_bib0052 article-title: SSD: single shot multibox detector – year: 2015 ident: 10.1016/j.patcog.2020.107461_bib0001 article-title: Very deep convolutional networks for large-scale image recognition – volume: 41 start-page: 2889 issue: 12 year: 2019 ident: 10.1016/j.patcog.2020.107461_bib0036 article-title: Holistic CNN compression via low-rank decomposition with knowledge transfer publication-title: IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) doi: 10.1109/TPAMI.2018.2873305 – start-page: 5058 year: 2017 ident: 10.1016/j.patcog.2020.107461_bib0002 article-title: ThiNet: a filter level pruning method for deep neural network compression – ident: 10.1016/j.patcog.2020.107461_bib0034 – year: 2018 ident: 10.1016/j.patcog.2020.107461_bib0025 article-title: Learning intrinsic sparse structures within long short-term memory – start-page: 2074 year: 2016 ident: 10.1016/j.patcog.2020.107461_bib0014 article-title: Learning structured sparsity in deep neural networks – year: 2017 ident: 10.1016/j.patcog.2020.107461_bib0042 article-title: Pruning convolutional neural networks for resource efficient inference – volume: 9908 start-page: 525 year: 2016 ident: 10.1016/j.patcog.2020.107461_bib0038 article-title: XNOR-Net: ImageNet classification using binary convolutional neural networks – start-page: 2148 year: 2013 ident: 10.1016/j.patcog.2020.107461_bib0009 article-title: Predicting parameters in deep learning – volume: 88 start-page: 303 issue: 2 year: 2010 ident: 10.1016/j.patcog.2020.107461_bib0050 article-title: The PASCAL visual object classes (VOC) challenge publication-title: Int. J. Comput. Vis. (IJCV) doi: 10.1007/s11263-009-0275-4 – year: 2019 ident: 10.1016/j.patcog.2020.107461_bib0053 article-title: Rethinking the value of network pruning – volume: 72 start-page: 72 year: 2017 ident: 10.1016/j.patcog.2020.107461_bib0013 article-title: Building fast and compact convolutional neural networks for offline handwritten chinese character recognition publication-title: Pattern Recognit. (PR) doi: 10.1016/j.patcog.2017.06.032 – year: 2019 ident: 10.1016/j.patcog.2020.107461_bib0035 article-title: Dynamic channel pruning: feature boosting and suppression – volume: 11220 start-page: 317 year: 2018 ident: 10.1016/j.patcog.2020.107461_bib0005 article-title: Data-driven sparse structure selection for deep neural networks – volume: 88 start-page: 272 year: 2019 ident: 10.1016/j.patcog.2020.107461_bib0011 article-title: LightweightNet: toward fast and lightweight convolutional neural networks via architecture distillation publication-title: Pattern Recognit. (PR) doi: 10.1016/j.patcog.2018.10.029 – start-page: 2425 year: 2018 ident: 10.1016/j.patcog.2020.107461_bib0033 article-title: Accelerating convolutional networks via global & dynamic filter pruning – volume: 86 start-page: 2278 issue: 11 year: 1998 ident: 10.1016/j.patcog.2020.107461_bib0043 article-title: Gradient-based learning applied to document recognition publication-title: Proc. IEEE doi: 10.1109/5.726791 – start-page: 1269 year: 2014 ident: 10.1016/j.patcog.2020.107461_bib0008 article-title: Exploiting linear structure within convolutional networks for efficient evaluation – start-page: 3290 year: 2017 ident: 10.1016/j.patcog.2020.107461_bib0020 article-title: Bayesian compression for deep learning – start-page: 2755 year: 2017 ident: 10.1016/j.patcog.2020.107461_bib0017 article-title: Learning efficient convolutional networks through network slimming – start-page: 1389 year: 2017 ident: 10.1016/j.patcog.2020.107461_bib0016 article-title: Channel pruning for accelerating very deep neural networks – start-page: 856 year: 2017 ident: 10.1016/j.patcog.2020.107461_bib0021 article-title: Compression-aware training of deep networks – year: 2016 ident: 10.1016/j.patcog.2020.107461_bib0024 article-title: Deep compression: compressing deep neural networks with pruning, trained quantization and Huffman coding – year: 2011 ident: 10.1016/j.patcog.2020.107461_bib0018 article-title: The Caltech-UCSD Birds-200–2011 Dataset – year: 2019 ident: 10.1016/j.patcog.2020.107461_bib0049 article-title: Slimmable neural networks – volume: 38 start-page: 1943 issue: 10 year: 2016 ident: 10.1016/j.patcog.2020.107461_bib0048 article-title: Accelerating very deep convolutional networks for classification and detection publication-title: IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) doi: 10.1109/TPAMI.2015.2502579 – start-page: 4820 year: 2016 ident: 10.1016/j.patcog.2020.107461_bib0010 article-title: Quantized convolutional neural networks for mobile devices – start-page: 4510 year: 2018 ident: 10.1016/j.patcog.2020.107461_bib0041 article-title: MobileNetV2: inverted residuals and linear bottlenecks |
| SSID | ssj0017142 |
| Score | 2.662579 |
| Snippet | •Filter selection and model fine-tuning are integrated into a single end-to-end trainable framework.•Adaptive compression ratio and multi-layer... |
| SourceID | crossref elsevier |
| SourceType | Enrichment Source Index Database Publisher |
| StartPage | 107461 |
| SubjectTerms | CNN acceleration Model compression Neural network pruning |
| Title | AutoPruner: An end-to-end trainable filter pruning method for efficient deep model inference |
| URI | https://dx.doi.org/10.1016/j.patcog.2020.107461 |
| Volume | 107 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV1LS8QwEA7LevHiW1wfSw5e46ZJmrTeyuKyKi4eXNiDUJo2kYq0Zele_e1m-hAFUfBUGibQTqfzIPPNh9AlZwm1KqEk86wmIglDkmRaERV4yrPUuHwOsMMPCzlfiruVvxqgaY-FgbbKzve3Pr3x1t3KpNPmpMpzwPjC2EEgAOcU-rQAwS4UWPnV-2ebB_B7txPDuUdAtIfPNT1elXN35YurEhksKSG9n8PTl5Az20M7Xa6Io_Zx9tHAFAdot-dhwN1veYieo01dPq43hVlf46jApshIXRJ3wQ0DBKCjsM3hXBxXTsxFK9wyR2OXsmLTTJFwwQdnxlS44cbBeQ8EPELL2c3TdE461gSScsVqommYZWGiZJgC255IwyC0VHiWmcBVD5RaJmEKXOoyP6l5IGXAUqs9oXyZuvKBH6NhURbmBGHfdyJMcmqkFYpJHbpqJdGSaZcEBdqOEO-VFafdSHF4r7e47x17jVsVx6DiuFXxCJHPXVU7UuMPedV_h_ibacTO6_-68_TfO8_QNty1oMNzNKzXG3Phso9ajxvzGqOt6PZ-vvgAwPLXsg |
| linkProvider | Elsevier |
| linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV1LS8NAEB5qe9CLb7E-9-B16WaT7ibeSrG09oGHFnoQQjbZlYqkoaT_39k8ioIoeApsZiCZbObBzvcNwIPLI2ZkxGjiGEW9KAholChJpe9IxzCN-ZzFDk9nYrjwnpfdZQP6NRbGtlVWvr_06YW3rlY6lTU72WplMb6WdtAOAHeZ7dPag5Zlp-o2odUbjYez3WGCdLySNNx1qJWuEXRFm1eGHm_9hoUit0vSE87PEepL1Bkcw2GVLpJe-UQn0NDpKRzVoxhI9WeewWtvm69fNttUbx5JLyU6TWi-pnghxRAIC5AiZmWPxkmGYhiwSDk8mmDWSnRBJIHxhyRaZ6QYj0NWNRbwHBaDp3l_SKvBCTR2Jc-pYkGSBJEUQWwH7nlx4AeGeY7h2scCgjHDhSWCizH5E8r1hfB5bJSDxhMxVhDuBTTTdaovgXS7KMKFy7QwnuRCBViwREpwhXmQr0wb3NpYYVyxitv3-gjr9rH3sDRxaE0cliZuA91pZSWrxh_ysv4O4bfdEaLj_1Xz6t-a97A_nE8n4WQ0G1_Dgb1TYhBvoJlvtvoWk5Fc3VWb7RM3Y9pj |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=AutoPruner%3A+An+end-to-end+trainable+filter+pruning+method+for+efficient+deep+model+inference&rft.jtitle=Pattern+recognition&rft.au=Luo%2C+Jian-Hao&rft.au=Wu%2C+Jianxin&rft.date=2020-11-01&rft.issn=0031-3203&rft.volume=107&rft.spage=107461&rft_id=info:doi/10.1016%2Fj.patcog.2020.107461&rft.externalDBID=n%2Fa&rft.externalDocID=10_1016_j_patcog_2020_107461 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0031-3203&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0031-3203&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0031-3203&client=summon |