Exploring the rate-distortion-complexity optimization in neural image compression
Despite a short history, neural image codecs have been shown to surpass classical image codecs in terms of rate–distortion performance. However, most of them suffer from significantly longer decoding times, which hinders the practical applications of neural image codecs. This issue is especially pro...
Saved in:
| Published in | Journal of visual communication and image representation Vol. 105; p. 104294 |
|---|---|
| Main Authors | , , , |
| Format | Journal Article |
| Language | English |
| Published |
Elsevier Inc
01.12.2024
|
| Subjects | |
| Online Access | Get full text |
| ISSN | 1047-3203 |
| DOI | 10.1016/j.jvcir.2024.104294 |
Cover
| Abstract | Despite a short history, neural image codecs have been shown to surpass classical image codecs in terms of rate–distortion performance. However, most of them suffer from significantly longer decoding times, which hinders the practical applications of neural image codecs. This issue is especially pronounced when employing an effective yet time-consuming autoregressive context model since it would increase entropy decoding time by orders of magnitude. In this paper, unlike most previous works that pursue optimal RD performance while temporally overlooking the coding complexity, we make a systematical investigation on the rate–distortion-complexity (RDC) optimization in neural image compression. By quantifying the decoding complexity as a factor in the optimization goal, we are now able to precisely control the RDC trade-off and then demonstrate how the rate–distortion performance of neural image codecs could adapt to various complexity demands. Going beyond the investigation of RDC optimization, a variable-complexity neural codec is designed to leverage the spatial dependencies adaptively according to industrial demands, which supports fine-grained complexity adjustment by balancing the RDC tradeoff. By implementing this scheme in a powerful base model, we demonstrate the feasibility and flexibility of RDC optimization for neural image codecs. |
|---|---|
| AbstractList | Despite a short history, neural image codecs have been shown to surpass classical image codecs in terms of rate–distortion performance. However, most of them suffer from significantly longer decoding times, which hinders the practical applications of neural image codecs. This issue is especially pronounced when employing an effective yet time-consuming autoregressive context model since it would increase entropy decoding time by orders of magnitude. In this paper, unlike most previous works that pursue optimal RD performance while temporally overlooking the coding complexity, we make a systematical investigation on the rate–distortion-complexity (RDC) optimization in neural image compression. By quantifying the decoding complexity as a factor in the optimization goal, we are now able to precisely control the RDC trade-off and then demonstrate how the rate–distortion performance of neural image codecs could adapt to various complexity demands. Going beyond the investigation of RDC optimization, a variable-complexity neural codec is designed to leverage the spatial dependencies adaptively according to industrial demands, which supports fine-grained complexity adjustment by balancing the RDC tradeoff. By implementing this scheme in a powerful base model, we demonstrate the feasibility and flexibility of RDC optimization for neural image codecs. |
| ArticleNumber | 104294 |
| Author | Chen, Zhibo Feng, Runsen Guo, Zongyu Gao, Yixin |
| Author_xml | – sequence: 1 givenname: Yixin surname: Gao fullname: Gao, Yixin – sequence: 2 givenname: Runsen orcidid: 0000-0001-6608-0785 surname: Feng fullname: Feng, Runsen – sequence: 3 givenname: Zongyu surname: Guo fullname: Guo, Zongyu – sequence: 4 givenname: Zhibo orcidid: 0000-0002-8525-5066 surname: Chen fullname: Chen, Zhibo email: chenzhibo@ustc.edu.cn |
| BookMark | eNp9kM1OwzAQhH0oEm3hCbj4BVL8m-ADB1SVH6kSQoKz5Tib4ii1I9tULU9PQjlz2tXMzmr0LdDMBw8I3VCyooSWt92qO1gXV4wwMSqCKTFD83GpCs4Iv0SLlDpCCFdczNHb5jj0ITq_w_kTcDQZisalHGJ2wRc27Iceji6fcBiy27tvM-nYeezhK5oeu73ZAZ7uIqQ0elfoojV9guu_uUQfj5v39XOxfX16WT9sC8skzwUwWhpm60byylhKWKlqIe2d5A1XraqoEbUByUwpuLQNgLRVrUgFQqmScOBLxM9_bQwpRWj1EMcy8aQp0RMJ3elfEnoioc8kxtT9OQVjtYODqJN14C00LoLNugnu3_wPPO1uIA |
| Cites_doi | 10.1109/TIP.2021.3058615 10.1109/ICCV.2019.00324 10.1109/CVPR46437.2021.01453 10.1109/CVPR42600.2020.00796 10.1109/TCSVT.2018.2880492 10.1109/TIP.2020.2982832 10.1109/TCSVT.2022.3146061 10.1109/TCSVT.2021.3053635 10.1109/TIP.2023.3266165 10.1109/CVPR52688.2022.00943 10.1109/TIP.2016.2601264 10.1109/CVPR52688.2022.00563 10.1109/30.125072 10.1109/TIP.2020.2985225 10.1109/JSTSP.2020.3034501 10.1109/TIP.2014.2341927 10.1109/TBC.2017.2711142 10.1109/TCSVT.2021.3119660 10.1109/TIP.2021.3083447 10.1109/TCSVT.2013.2249017 10.1109/TCSVT.2012.2221191 10.1109/TCSVT.2021.3108671 10.1109/CVPR52688.2022.01553 10.1007/s00034-019-01110-4 10.1007/s11554-020-00996-7 10.1109/TCSVT.2021.3089491 10.1109/TMM.2021.3052348 10.1109/TII.2018.2844214 10.1109/ICCV48922.2021.00238 10.1109/TCSVT.2019.2904198 |
| ContentType | Journal Article |
| Copyright | 2024 Elsevier Inc. |
| Copyright_xml | – notice: 2024 Elsevier Inc. |
| DBID | AAYXX CITATION |
| DOI | 10.1016/j.jvcir.2024.104294 |
| DatabaseName | CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Journalism & Communications Engineering |
| ExternalDocumentID | 10_1016_j_jvcir_2024_104294 S1047320324002505 |
| GroupedDBID | --K --M .DC .~1 0R~ 1B1 1~. 1~5 29L 4.4 457 4G. 53G 5GY 5VS 7-5 71M 8P~ 9JN AACTN AAEDT AAEDW AAIKJ AAKOC AALRI AAOAW AAQFI AAQXK AAXKI AAXUO AAYFN ABBOA ABFNM ABJNI ABMAC ABWVN ABXDB ACDAQ ACGFS ACNNM ACRLP ACRPL ACZNC ADBBV ADEZE ADFGL ADJOM ADMHC ADMUD ADNMO ADTZH AEBSH AECPX AEKER AENEX AFJKZ AFKWA AFTJW AGHFR AGUBO AGYEJ AHHHB AHJVU AHZHX AIALX AIEXJ AIKHN AITUG AJOXV AKRWK ALMA_UNASSIGNED_HOLDINGS AMFUW AMRAJ AOUOD ASPBG AVWKF AXJTR AZFZN BJAXD BKOJK BLXMC CAG COF CS3 DM4 DU5 EBS EFBJH EJD EO8 EO9 EP2 EP3 F5P FDB FEDTE FGOYB FIRID FNPLU FYGXN G-2 G-Q GBLVA GBOLZ HLZ HVGLF HZ~ IHE J1W JJJVA KOM LG5 LX9 M41 MO0 N9A O-L O9- OAUVE OZT P-8 P-9 P2P PC. Q38 R2- RIG ROL RPZ SBC SDF SDG SDP SES SEW SPC SPCBC SST SSV SSZ T5K WH7 WUQ XPP YQT ZMT ZU3 ~G- AATTM AAYWO AAYXX ACVFH ADCNI AEIPS AEUPX AFPUW AGCQF AGQPQ AIGII AIIUN AKBMS AKYEP ANKPU APXCP CITATION EFKBS EFLBG |
| ID | FETCH-LOGICAL-c253t-e216a2cbd537ac10269b45c853d39f971a4bae52a6435cdee5c7b907e499603e3 |
| IEDL.DBID | AIKHN |
| ISSN | 1047-3203 |
| IngestDate | Wed Sep 10 06:22:37 EDT 2025 Sat Dec 21 15:59:34 EST 2024 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Keywords | Rate-distortion-complexity optimization Variable-complexity Neural image compression |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c253t-e216a2cbd537ac10269b45c853d39f971a4bae52a6435cdee5c7b907e499603e3 |
| ORCID | 0000-0002-8525-5066 0000-0001-6608-0785 |
| ParticipantIDs | crossref_primary_10_1016_j_jvcir_2024_104294 elsevier_sciencedirect_doi_10_1016_j_jvcir_2024_104294 |
| PublicationCentury | 2000 |
| PublicationDate | December 2024 2024-12-00 |
| PublicationDateYYYYMMDD | 2024-12-01 |
| PublicationDate_xml | – month: 12 year: 2024 text: December 2024 |
| PublicationDecade | 2020 |
| PublicationTitle | Journal of visual communication and image representation |
| PublicationYear | 2024 |
| Publisher | Elsevier Inc |
| Publisher_xml | – name: Elsevier Inc |
| References | Liu, Yu, Gao, Chen, Ji, Wang (b35) 2016; 25 Cho, Kim (b36) 2013; 23 M. Song, J. Choi, B. Han, Variable-Rate Deep Image Compression through Spatially-Adaptive Feature Transform, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 2380–2389. E. Jang, S. Gu, B. Poole, Categorical reparameterization with gumbel-softmax, in: International Conference on Learning Representations, 2017. Feng, Liu, Liu, Li, Wu (b42) 2023 Ballé, Chou, Minnen, Singh, Johnston, Agustsson, Hwang, Toderici (b46) 2020; 15 Minnen, Singh (b22) 2020 Dong, Shen, Yu, Yang (b5) 2021; 24 J. Lee, S. Cho, S.-K. Beack, Context-adaptive Entropy Model for End-to-end Optimized Image Compression, in: The 7th Int. Conf. on Learning Representations, 2019. Saldanha, Sanchez, Marcon, Agostini (b41) 2021; 32 Guo, Zhang, Feng, Chen (b20) 2021; 32 Z. Xie, Z. Zhang, Y. Cao, Y. Lin, J. Bao, Z. Yao, Q. Dai, H. Hu, Simmim: A simple framework for masked image modeling, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 9653–9663. Sullivan, Ohm, Han, Wiegand (b1) 2012; 22 Shen, Zhang, Liu (b3) 2014; 23 Wallace (b29) 1992; 38 N. Asuni, A. Giachetti, TESTIMAGES: a Large-scale Archive for Testing Visual Devices and Basic Image Processing Algorithms, in: STAG, 2014, pp. 63–70. Rabbani, Joshi (b15) 2002; 17 Y. Choi, M. El-Khamy, J. Lee, Variable rate deep image compression with a conditional autoencoder, in: Proceedings of the IEEE International Conference on Computer Vision, 2019, pp. 3146–3154. K. He, X. Chen, S. Xie, Y. Li, P. Dollár, R. Girshick, Masked autoencoders are scalable vision learners, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 16000–16009. Kodak lossless true color image suite D.P. Kingma, J. Ba, Adam: A method for stochastic optimization, in: International Conference on Learning Representations, 2015. Wu, Li, Zhang, Jin, Chen (b44) 2021; 32 J. Ballé, V. Laparra, E.P. Simoncelli, End-to-end Optimized Image Compression, in: International Conference on Learning Representations, 2017. Cai, Chen, Zhang, Gao (b50) 2019; 29 Chen, Liu, Ma, Shen, Cao, Wang (b24) 2021; 30 Ballé, Laparra, Simoncelli (b47) 2016 . Van Oord, Kalchbrenner, Kavukcuoglu (b25) 2016 H. Bao, L. Dong, S. Piao, F. Wei, BEiT: BERT Pre-Training of Image Transformers, in: International Conference on Learning Representations, 2022. Guo, Zhang, Feng, Chen (b14) 2021; Vol. 139 Huang, Song, Xie, Izquierdo, Zhang (b40) 2021; 31 Van den Oord, Kalchbrenner, Espeholt, Vinyals, Graves (b48) 2016; 29 Zhang, Kwong, Zhao, Ip (b37) 2018; 15 Hosseini, Pakdaman, Hashemi, Ghanbari (b43) 2021; 18 D. Minnen, J. Ballé, G.D. Toderici, Joint autoregressive and hierarchical priors for learned image compression, in: Advances in Neural Information Processing Systems, 2018, pp. 10794–10803. Pan, Guo, Chen (b11) 2021 Cui, Wang, Gao, Guo, Feng, Bai (b52) 2021 Z. Cheng, H. Sun, M. Takeuchi, J. Katto, Learned image compression with discretized gaussian mixture likelihoods and attention modules, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 7939–7948. Li, Ma, You, Zhang, Zuo (b21) 2020; 29 Bellard (b16) 2015 Tang, Jing, Zeng, Fan (b32) 2019 Kuanar, Rao, Bilas, Bredow (b7) 2019; 38 Shi, Gao, Chen (b4) 2019 Min, Cheung (b30) 2014; 25 Wu, Shi, Chen (b8) 2022; 32 Y. Qian, Z. Tan, X. Sun, M. Lin, D. Li, Z. Sun, H. Li, R. Jin, Learning Accurate Entropy Model with Global Reference for Image Compression, in: International Conference on Learning Representations, 2021. D. He, Z. Yang, W. Peng, R. Ma, H. Qin, Y. Wang, Elic: Efficient learned image compression with unevenly grouped space-channel contextual adaptive coding, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 5718–5727. J.-R. Ohm, G.J. Sullivan, Versatile video coding–towards the next generation of video compression, in: Picture Coding Symposium, Vol. 2018, 2018. J. Zhou, Multi-scale and Context-adaptive Entropy Model for Image Compression, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2019. Li, Xu, Tang, Chen, Xing (b38) 2021; 30 J. Ballé, D. Minnen, S. Singh, S.J. Hwang, N. Johnston, Variational image compression with a scale hyperprior, in: International Conference on Learning Representations, 2018. Chen, Shi, Li (b39) 2020; 29 Zhu, Zhang, Pan, Wang, Kwong, Peng (b6) 2017; 63 Saldanha, Sanchez, Marcon, Agostini (b34) 2021 Yang, Shen, Dong, Ding, An, Jiang (b31) 2019; 30 D. He, Y. Zheng, B. Sun, Y. Wang, H. Qin, Checkerboard context model for efficient learned image compression, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 14771–14780. Lin, Maire, Belongie, Hays, Perona, Ramanan, Dollár, Zitnick (b54) 2014 Cui, Zhang, Gu, Zhang, Ma (b33) 2020 10.1016/j.jvcir.2024.104294_b27 10.1016/j.jvcir.2024.104294_b28 Sullivan (10.1016/j.jvcir.2024.104294_b1) 2012; 22 Cui (10.1016/j.jvcir.2024.104294_b33) 2020 Kuanar (10.1016/j.jvcir.2024.104294_b7) 2019; 38 Van Oord (10.1016/j.jvcir.2024.104294_b25) 2016 Chen (10.1016/j.jvcir.2024.104294_b24) 2021; 30 Lin (10.1016/j.jvcir.2024.104294_b54) 2014 Guo (10.1016/j.jvcir.2024.104294_b14) 2021; Vol. 139 Yang (10.1016/j.jvcir.2024.104294_b31) 2019; 30 Liu (10.1016/j.jvcir.2024.104294_b35) 2016; 25 Guo (10.1016/j.jvcir.2024.104294_b20) 2021; 32 10.1016/j.jvcir.2024.104294_b9 Pan (10.1016/j.jvcir.2024.104294_b11) 2021 Chen (10.1016/j.jvcir.2024.104294_b39) 2020; 29 Saldanha (10.1016/j.jvcir.2024.104294_b41) 2021; 32 10.1016/j.jvcir.2024.104294_b2 Min (10.1016/j.jvcir.2024.104294_b30) 2014; 25 Rabbani (10.1016/j.jvcir.2024.104294_b15) 2002; 17 Minnen (10.1016/j.jvcir.2024.104294_b22) 2020 Cui (10.1016/j.jvcir.2024.104294_b52) 2021 Van den Oord (10.1016/j.jvcir.2024.104294_b48) 2016; 29 Zhu (10.1016/j.jvcir.2024.104294_b6) 2017; 63 Li (10.1016/j.jvcir.2024.104294_b38) 2021; 30 Shen (10.1016/j.jvcir.2024.104294_b3) 2014; 23 Huang (10.1016/j.jvcir.2024.104294_b40) 2021; 31 Feng (10.1016/j.jvcir.2024.104294_b42) 2023 Wu (10.1016/j.jvcir.2024.104294_b8) 2022; 32 Hosseini (10.1016/j.jvcir.2024.104294_b43) 2021; 18 Ballé (10.1016/j.jvcir.2024.104294_b46) 2020; 15 10.1016/j.jvcir.2024.104294_b45 10.1016/j.jvcir.2024.104294_b49 Saldanha (10.1016/j.jvcir.2024.104294_b34) 2021 Shi (10.1016/j.jvcir.2024.104294_b4) 2019 Wu (10.1016/j.jvcir.2024.104294_b44) 2021; 32 Bellard (10.1016/j.jvcir.2024.104294_b16) 2015 10.1016/j.jvcir.2024.104294_b51 Cai (10.1016/j.jvcir.2024.104294_b50) 2019; 29 10.1016/j.jvcir.2024.104294_b53 10.1016/j.jvcir.2024.104294_b10 10.1016/j.jvcir.2024.104294_b55 10.1016/j.jvcir.2024.104294_b12 10.1016/j.jvcir.2024.104294_b56 10.1016/j.jvcir.2024.104294_b13 10.1016/j.jvcir.2024.104294_b57 10.1016/j.jvcir.2024.104294_b17 Cho (10.1016/j.jvcir.2024.104294_b36) 2013; 23 10.1016/j.jvcir.2024.104294_b18 10.1016/j.jvcir.2024.104294_b19 Zhang (10.1016/j.jvcir.2024.104294_b37) 2018; 15 Wallace (10.1016/j.jvcir.2024.104294_b29) 1992; 38 Ballé (10.1016/j.jvcir.2024.104294_b47) 2016 Dong (10.1016/j.jvcir.2024.104294_b5) 2021; 24 Tang (10.1016/j.jvcir.2024.104294_b32) 2019 Li (10.1016/j.jvcir.2024.104294_b21) 2020; 29 10.1016/j.jvcir.2024.104294_b23 10.1016/j.jvcir.2024.104294_b26 |
| References_xml | – volume: 29 start-page: 5431 year: 2020 end-page: 5446 ident: b39 article-title: Learned fast HEVC intra coding publication-title: IEEE Trans. Image Process. – start-page: 740 year: 2014 end-page: 755 ident: b54 article-title: Microsoft coco: Common objects in context publication-title: Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13 – reference: J. Zhou, Multi-scale and Context-adaptive Entropy Model for Image Compression, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2019. – volume: 30 start-page: 1668 year: 2019 end-page: 1682 ident: b31 article-title: Low-complexity CTU partition structure decision and fast intra mode decision for versatile video coding publication-title: IEEE Trans. Circuits Syst. Video Technol. – start-page: 1747 year: 2016 end-page: 1756 ident: b25 article-title: Pixel recurrent neural networks publication-title: International Conference on Machine Learning – volume: Vol. 139 start-page: 3920 year: 2021 end-page: 3929 ident: b14 article-title: Soft then hard: Rethinking the quantization in neural image compression publication-title: Proceedings of the 38th International Conference on Machine Learning – year: 2023 ident: b42 article-title: Partition map prediction for fast block partitioning in VVC intra-frame coding publication-title: IEEE Trans. Image Process. – reference: D. Minnen, J. Ballé, G.D. Toderici, Joint autoregressive and hierarchical priors for learned image compression, in: Advances in Neural Information Processing Systems, 2018, pp. 10794–10803. – volume: 15 start-page: 339 year: 2020 end-page: 353 ident: b46 article-title: Nonlinear transform coding publication-title: IEEE J. Sel. Top. Sign. Proces. – reference: D. He, Z. Yang, W. Peng, R. Ma, H. Qin, Y. Wang, Elic: Efficient learned image compression with unevenly grouped space-channel contextual adaptive coding, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 5718–5727. – volume: 25 start-page: 892 year: 2014 end-page: 896 ident: b30 article-title: A fast CU size decision algorithm for the HEVC intra encoder publication-title: IEEE Trans. Circuits Syst. Video Technol. – reference: D. He, Y. Zheng, B. Sun, Y. Wang, H. Qin, Checkerboard context model for efficient learned image compression, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 14771–14780. – volume: 31 start-page: 4454 year: 2021 end-page: 4469 ident: b40 article-title: Modeling acceleration properties for flexible INTRA HEVC complexity control publication-title: IEEE Trans. Circuits Syst. Video Technol. – year: 2015 ident: b16 article-title: BPG image format – volume: 24 start-page: 400 year: 2021 end-page: 414 ident: b5 article-title: Fast intra mode decision algorithm for versatile video coding publication-title: IEEE Trans. Multimed. – start-page: 10527 year: 2021 end-page: 10536 ident: b52 article-title: Asymmetric gained deep image compression with continuous rate adaptation publication-title: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition – reference: J. Lee, S. Cho, S.-K. Beack, Context-adaptive Entropy Model for End-to-end Optimized Image Compression, in: The 7th Int. Conf. on Learning Representations, 2019. – reference: Kodak lossless true color image suite, – reference: H. Bao, L. Dong, S. Piao, F. Wei, BEiT: BERT Pre-Training of Image Transformers, in: International Conference on Learning Representations, 2022. – volume: 29 start-page: 3687 year: 2019 end-page: 3700 ident: b50 article-title: Efficient variable rate image compression with multi-scale decomposition network publication-title: IEEE Trans. Circuits Syst. Video Technol. – volume: 30 start-page: 3179 year: 2021 end-page: 3191 ident: b24 article-title: End-to-end learnt image compression via non-local attention optimization and improved context modeling publication-title: IEEE Trans. Image Process. – volume: 32 start-page: 2329 year: 2021 end-page: 2341 ident: b20 article-title: Causal contextual prediction for learned image compression publication-title: IEEE Trans. Circuits Syst. Video Technol. – reference: E. Jang, S. Gu, B. Poole, Categorical reparameterization with gumbel-softmax, in: International Conference on Learning Representations, 2017. – reference: Z. Cheng, H. Sun, M. Takeuchi, J. Katto, Learned image compression with discretized gaussian mixture likelihoods and attention modules, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 7939–7948. – start-page: 103 year: 2020 end-page: 112 ident: b33 article-title: Gradient-based early termination of CU partition in VVC intra coding publication-title: 2020 Data Compression Conference – volume: 32 start-page: 3947 year: 2021 end-page: 3960 ident: b41 article-title: Configurable fast block partitioning for VVC intra coding using light gradient boosting machine publication-title: IEEE Trans. Circuits Syst. Video Technol. – start-page: 3339 year: 2020 end-page: 3343 ident: b22 article-title: Channel-wise autoregressive entropy models for learned image compression publication-title: 2020 IEEE International Conference on Image Processing – volume: 32 start-page: 5638 year: 2022 end-page: 5649 ident: b8 article-title: HG-FCN: Hierarchical grid fully convolutional network for fast VVC intra coding publication-title: IEEE Trans. Circuits Syst. Video Technol. – reference: D.P. Kingma, J. Ba, Adam: A method for stochastic optimization, in: International Conference on Learning Representations, 2015. – volume: 22 start-page: 1649 year: 2012 end-page: 1668 ident: b1 article-title: Overview of the high efficiency video coding (HEVC) standard publication-title: IEEE Trans. Circuits Syst. Video Technol. – reference: Y. Qian, Z. Tan, X. Sun, M. Lin, D. Li, Z. Sun, H. Li, R. Jin, Learning Accurate Entropy Model with Global Reference for Image Compression, in: International Conference on Learning Representations, 2021. – start-page: 1 year: 2019 end-page: 4 ident: b32 article-title: Adaptive CU split decision with pooling-variable CNN for VVC intra encoding publication-title: 2019 IEEE Visual Communications and Image Processing – volume: 23 start-page: 4232 year: 2014 end-page: 4241 ident: b3 article-title: Effective CU size decision for HEVC intracoding publication-title: IEEE Trans. Image Process. – reference: M. Song, J. Choi, B. Han, Variable-Rate Deep Image Compression through Spatially-Adaptive Feature Transform, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 2380–2389. – start-page: 1 year: 2021 end-page: 5 ident: b34 article-title: Learning-based complexity reduction scheme for VVC intra-frame prediction publication-title: 2021 International Conference on Visual Communications and Image Processing – volume: 17 start-page: 3 year: 2002 end-page: 48 ident: b15 article-title: An overview of the JPEG 2000 still image compression standard publication-title: Signal Process.: Image Commun. – volume: 38 start-page: 5081 year: 2019 end-page: 5102 ident: b7 article-title: Adaptive CU mode selection in HEVC intra prediction: A deep learning approach publication-title: Circuits Systems Signal Process. – start-page: 1 year: 2019 end-page: 5 ident: b4 article-title: Asymmetric-kernel CNN based fast CTU partition for HEVC intra coding publication-title: 2019 IEEE International Symposium on Circuits and Systems – volume: 18 start-page: 603 year: 2021 end-page: 618 ident: b43 article-title: Fine-grain complexity control of HEVC intra prediction in battery-powered video codecs publication-title: J. Real-Time Image Process. – reference: N. Asuni, A. Giachetti, TESTIMAGES: a Large-scale Archive for Testing Visual Devices and Basic Image Processing Algorithms, in: STAG, 2014, pp. 63–70. – volume: 23 start-page: 1555 year: 2013 end-page: 1564 ident: b36 article-title: Fast CU splitting and pruning for suboptimal CU partitioning in HEVC intra coding publication-title: IEEE Trans. Circuits Syst. Video Technol. – volume: 63 start-page: 547 year: 2017 end-page: 561 ident: b6 article-title: Binary and multi-class learning based low complexity optimization for HEVC encoding publication-title: IEEE Trans. Broadcast. – reference: K. He, X. Chen, S. Xie, Y. Li, P. Dollár, R. Girshick, Masked autoencoders are scalable vision learners, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 16000–16009. – volume: 15 start-page: 1437 year: 2018 end-page: 1449 ident: b37 article-title: Complexity control in the HEVC intracoding for industrial video applications publication-title: IEEE Trans. Ind. Inform. – reference: Y. Choi, M. El-Khamy, J. Lee, Variable rate deep image compression with a conditional autoencoder, in: Proceedings of the IEEE International Conference on Computer Vision, 2019, pp. 3146–3154. – volume: 29 year: 2016 ident: b48 article-title: Conditional image generation with pixelcnn decoders publication-title: Adv. Neural Inf. Process. Syst. – volume: 38 start-page: xviii year: 1992 end-page: xxxiv ident: b29 article-title: The JPEG still picture compression standard publication-title: IEEE Trans. Consum. Electron. – start-page: 1 year: 2016 end-page: 5 ident: b47 article-title: End-to-end optimization of nonlinear transform codes for perceptual quality publication-title: Picture Coding Symposium (PCS), 2016 – volume: 32 start-page: 3978 year: 2021 end-page: 3990 ident: b44 article-title: Learned block-based hybrid image compression publication-title: IEEE Trans. Circuits Syst. Video Technol. – reference: Z. Xie, Z. Zhang, Y. Cao, Y. Lin, J. Bao, Z. Yao, Q. Dai, H. Hu, Simmim: A simple framework for masked image modeling, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 9653–9663. – reference: J. Ballé, D. Minnen, S. Singh, S.J. Hwang, N. Johnston, Variational image compression with a scale hyperprior, in: International Conference on Learning Representations, 2018. – reference: J. Ballé, V. Laparra, E.P. Simoncelli, End-to-end Optimized Image Compression, in: International Conference on Learning Representations, 2017. – reference: J.-R. Ohm, G.J. Sullivan, Versatile video coding–towards the next generation of video compression, in: Picture Coding Symposium, Vol. 2018, 2018. – volume: 30 start-page: 5377 year: 2021 end-page: 5390 ident: b38 article-title: DeepQTMT: A deep learning approach for fast QTMT-based CU partition of intra-mode VVC publication-title: IEEE Trans. Image Process. – volume: 25 start-page: 5088 year: 2016 end-page: 5103 ident: b35 article-title: CU partition mode decision for HEVC hardwired intra encoder using convolution neural network publication-title: IEEE Trans. Image Process. – reference: . – volume: 29 start-page: 5900 year: 2020 end-page: 5911 ident: b21 article-title: Efficient and effective context-based convolutional entropy modeling for image compression publication-title: IEEE Trans. Image Process. – start-page: 1 year: 2021 end-page: 5 ident: b11 article-title: Analyzing time complexity of practical learned image compression models publication-title: 2021 International Conference on Visual Communications and Image Processing – start-page: 1 year: 2019 ident: 10.1016/j.jvcir.2024.104294_b32 article-title: Adaptive CU split decision with pooling-variable CNN for VVC intra encoding – volume: 30 start-page: 3179 year: 2021 ident: 10.1016/j.jvcir.2024.104294_b24 article-title: End-to-end learnt image compression via non-local attention optimization and improved context modeling publication-title: IEEE Trans. Image Process. doi: 10.1109/TIP.2021.3058615 – ident: 10.1016/j.jvcir.2024.104294_b17 – volume: 17 start-page: 3 issue: 1 year: 2002 ident: 10.1016/j.jvcir.2024.104294_b15 article-title: An overview of the JPEG 2000 still image compression standard publication-title: Signal Process.: Image Commun. – ident: 10.1016/j.jvcir.2024.104294_b51 doi: 10.1109/ICCV.2019.00324 – start-page: 1 year: 2021 ident: 10.1016/j.jvcir.2024.104294_b34 article-title: Learning-based complexity reduction scheme for VVC intra-frame prediction – ident: 10.1016/j.jvcir.2024.104294_b23 doi: 10.1109/CVPR46437.2021.01453 – ident: 10.1016/j.jvcir.2024.104294_b13 doi: 10.1109/CVPR42600.2020.00796 – ident: 10.1016/j.jvcir.2024.104294_b55 – start-page: 103 year: 2020 ident: 10.1016/j.jvcir.2024.104294_b33 article-title: Gradient-based early termination of CU partition in VVC intra coding – ident: 10.1016/j.jvcir.2024.104294_b26 – start-page: 3339 year: 2020 ident: 10.1016/j.jvcir.2024.104294_b22 article-title: Channel-wise autoregressive entropy models for learned image compression – volume: 29 start-page: 3687 issue: 12 year: 2019 ident: 10.1016/j.jvcir.2024.104294_b50 article-title: Efficient variable rate image compression with multi-scale decomposition network publication-title: IEEE Trans. Circuits Syst. Video Technol. doi: 10.1109/TCSVT.2018.2880492 – volume: 25 start-page: 892 issue: 5 year: 2014 ident: 10.1016/j.jvcir.2024.104294_b30 article-title: A fast CU size decision algorithm for the HEVC intra encoder publication-title: IEEE Trans. Circuits Syst. Video Technol. – start-page: 1 year: 2016 ident: 10.1016/j.jvcir.2024.104294_b47 article-title: End-to-end optimization of nonlinear transform codes for perceptual quality – ident: 10.1016/j.jvcir.2024.104294_b18 – ident: 10.1016/j.jvcir.2024.104294_b9 – volume: 29 start-page: 5431 year: 2020 ident: 10.1016/j.jvcir.2024.104294_b39 article-title: Learned fast HEVC intra coding publication-title: IEEE Trans. Image Process. doi: 10.1109/TIP.2020.2982832 – ident: 10.1016/j.jvcir.2024.104294_b45 – ident: 10.1016/j.jvcir.2024.104294_b49 – start-page: 740 year: 2014 ident: 10.1016/j.jvcir.2024.104294_b54 article-title: Microsoft coco: Common objects in context – volume: 32 start-page: 5638 issue: 8 year: 2022 ident: 10.1016/j.jvcir.2024.104294_b8 article-title: HG-FCN: Hierarchical grid fully convolutional network for fast VVC intra coding publication-title: IEEE Trans. Circuits Syst. Video Technol. doi: 10.1109/TCSVT.2022.3146061 – volume: 31 start-page: 4454 issue: 11 year: 2021 ident: 10.1016/j.jvcir.2024.104294_b40 article-title: Modeling acceleration properties for flexible INTRA HEVC complexity control publication-title: IEEE Trans. Circuits Syst. Video Technol. doi: 10.1109/TCSVT.2021.3053635 – ident: 10.1016/j.jvcir.2024.104294_b56 – year: 2023 ident: 10.1016/j.jvcir.2024.104294_b42 article-title: Partition map prediction for fast block partitioning in VVC intra-frame coding publication-title: IEEE Trans. Image Process. doi: 10.1109/TIP.2023.3266165 – ident: 10.1016/j.jvcir.2024.104294_b28 doi: 10.1109/CVPR52688.2022.00943 – volume: 25 start-page: 5088 issue: 11 year: 2016 ident: 10.1016/j.jvcir.2024.104294_b35 article-title: CU partition mode decision for HEVC hardwired intra encoder using convolution neural network publication-title: IEEE Trans. Image Process. doi: 10.1109/TIP.2016.2601264 – volume: Vol. 139 start-page: 3920 year: 2021 ident: 10.1016/j.jvcir.2024.104294_b14 article-title: Soft then hard: Rethinking the quantization in neural image compression – ident: 10.1016/j.jvcir.2024.104294_b10 doi: 10.1109/CVPR52688.2022.00563 – volume: 38 start-page: xviii issue: 1 year: 1992 ident: 10.1016/j.jvcir.2024.104294_b29 article-title: The JPEG still picture compression standard publication-title: IEEE Trans. Consum. Electron. doi: 10.1109/30.125072 – ident: 10.1016/j.jvcir.2024.104294_b19 – volume: 29 start-page: 5900 year: 2020 ident: 10.1016/j.jvcir.2024.104294_b21 article-title: Efficient and effective context-based convolutional entropy modeling for image compression publication-title: IEEE Trans. Image Process. doi: 10.1109/TIP.2020.2985225 – volume: 15 start-page: 339 issue: 2 year: 2020 ident: 10.1016/j.jvcir.2024.104294_b46 article-title: Nonlinear transform coding publication-title: IEEE J. Sel. Top. Sign. Proces. doi: 10.1109/JSTSP.2020.3034501 – volume: 23 start-page: 4232 issue: 10 year: 2014 ident: 10.1016/j.jvcir.2024.104294_b3 article-title: Effective CU size decision for HEVC intracoding publication-title: IEEE Trans. Image Process. doi: 10.1109/TIP.2014.2341927 – ident: 10.1016/j.jvcir.2024.104294_b57 – volume: 63 start-page: 547 issue: 3 year: 2017 ident: 10.1016/j.jvcir.2024.104294_b6 article-title: Binary and multi-class learning based low complexity optimization for HEVC encoding publication-title: IEEE Trans. Broadcast. doi: 10.1109/TBC.2017.2711142 – volume: 32 start-page: 3978 issue: 6 year: 2021 ident: 10.1016/j.jvcir.2024.104294_b44 article-title: Learned block-based hybrid image compression publication-title: IEEE Trans. Circuits Syst. Video Technol. doi: 10.1109/TCSVT.2021.3119660 – volume: 30 start-page: 5377 year: 2021 ident: 10.1016/j.jvcir.2024.104294_b38 article-title: DeepQTMT: A deep learning approach for fast QTMT-based CU partition of intra-mode VVC publication-title: IEEE Trans. Image Process. doi: 10.1109/TIP.2021.3083447 – start-page: 1 year: 2021 ident: 10.1016/j.jvcir.2024.104294_b11 article-title: Analyzing time complexity of practical learned image compression models – volume: 23 start-page: 1555 issue: 9 year: 2013 ident: 10.1016/j.jvcir.2024.104294_b36 article-title: Fast CU splitting and pruning for suboptimal CU partitioning in HEVC intra coding publication-title: IEEE Trans. Circuits Syst. Video Technol. doi: 10.1109/TCSVT.2013.2249017 – volume: 22 start-page: 1649 issue: 12 year: 2012 ident: 10.1016/j.jvcir.2024.104294_b1 article-title: Overview of the high efficiency video coding (HEVC) standard publication-title: IEEE Trans. Circuits Syst. Video Technol. doi: 10.1109/TCSVT.2012.2221191 – ident: 10.1016/j.jvcir.2024.104294_b2 – volume: 32 start-page: 3947 issue: 6 year: 2021 ident: 10.1016/j.jvcir.2024.104294_b41 article-title: Configurable fast block partitioning for VVC intra coding using light gradient boosting machine publication-title: IEEE Trans. Circuits Syst. Video Technol. doi: 10.1109/TCSVT.2021.3108671 – start-page: 1 year: 2019 ident: 10.1016/j.jvcir.2024.104294_b4 article-title: Asymmetric-kernel CNN based fast CTU partition for HEVC intra coding – ident: 10.1016/j.jvcir.2024.104294_b27 doi: 10.1109/CVPR52688.2022.01553 – start-page: 10527 year: 2021 ident: 10.1016/j.jvcir.2024.104294_b52 article-title: Asymmetric gained deep image compression with continuous rate adaptation – volume: 38 start-page: 5081 year: 2019 ident: 10.1016/j.jvcir.2024.104294_b7 article-title: Adaptive CU mode selection in HEVC intra prediction: A deep learning approach publication-title: Circuits Systems Signal Process. doi: 10.1007/s00034-019-01110-4 – volume: 18 start-page: 603 year: 2021 ident: 10.1016/j.jvcir.2024.104294_b43 article-title: Fine-grain complexity control of HEVC intra prediction in battery-powered video codecs publication-title: J. Real-Time Image Process. doi: 10.1007/s11554-020-00996-7 – volume: 32 start-page: 2329 issue: 4 year: 2021 ident: 10.1016/j.jvcir.2024.104294_b20 article-title: Causal contextual prediction for learned image compression publication-title: IEEE Trans. Circuits Syst. Video Technol. doi: 10.1109/TCSVT.2021.3089491 – volume: 24 start-page: 400 year: 2021 ident: 10.1016/j.jvcir.2024.104294_b5 article-title: Fast intra mode decision algorithm for versatile video coding publication-title: IEEE Trans. Multimed. doi: 10.1109/TMM.2021.3052348 – volume: 15 start-page: 1437 issue: 3 year: 2018 ident: 10.1016/j.jvcir.2024.104294_b37 article-title: Complexity control in the HEVC intracoding for industrial video applications publication-title: IEEE Trans. Ind. Inform. doi: 10.1109/TII.2018.2844214 – volume: 29 year: 2016 ident: 10.1016/j.jvcir.2024.104294_b48 article-title: Conditional image generation with pixelcnn decoders publication-title: Adv. Neural Inf. Process. Syst. – ident: 10.1016/j.jvcir.2024.104294_b12 – start-page: 1747 year: 2016 ident: 10.1016/j.jvcir.2024.104294_b25 article-title: Pixel recurrent neural networks – ident: 10.1016/j.jvcir.2024.104294_b53 doi: 10.1109/ICCV48922.2021.00238 – volume: 30 start-page: 1668 issue: 6 year: 2019 ident: 10.1016/j.jvcir.2024.104294_b31 article-title: Low-complexity CTU partition structure decision and fast intra mode decision for versatile video coding publication-title: IEEE Trans. Circuits Syst. Video Technol. doi: 10.1109/TCSVT.2019.2904198 – year: 2015 ident: 10.1016/j.jvcir.2024.104294_b16 |
| SSID | ssj0003934 |
| Score | 2.4025834 |
| Snippet | Despite a short history, neural image codecs have been shown to surpass classical image codecs in terms of rate–distortion performance. However, most of them... |
| SourceID | crossref elsevier |
| SourceType | Index Database Publisher |
| StartPage | 104294 |
| SubjectTerms | Neural image compression Rate-distortion-complexity optimization Variable-complexity |
| Title | Exploring the rate-distortion-complexity optimization in neural image compression |
| URI | https://dx.doi.org/10.1016/j.jvcir.2024.104294 |
| Volume | 105 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV3PS8MwFH7M7aIH0ak4pyMH8WSd-dn1OIZj_hr-hN1KmmbQwbrhpnjybzdp0uFAPHhseSHlJXzvK_neF4BTJS9VZNVTWHdUYCreOJBUiICEckwxVpqn9kT3figGr-xmxEcV6JW9MFZW6bHfYXqB1v5N22ezPc-y9rM1GaD2AnDmCvkG1AiNBK9CrXt9OxiuAJlG7nDZmhLYAaX5UCHzmnyozPqCEmaPO0nEfi9QP4pOfwe2PVtEXfdBu1DReR22fngI1qHpg7LFFJ2htX6PxR48rjR2yDA9ZH0hgrRwBrEBQSEo15-GiaOZwY6pb8pEWY6s0aWZOpsawEE2zglm83147V-99AaBv0UhUITTZaAJFpKoJOU0lMrwCREljCtTplOzMlGIJUuk5kQabsJVqjVXYWJ-mTWzxi1U0wOo5rNcHwJiqaFzRKdchJKRxFBLJUXUSSKFqdIYN-C8TF08d2YZcakim8RFpmOb6dhlugGiTG-8tuaxgfO_Bh79d2ATNu2Tk6McQ3X59q5PDKlYJi3YuPjCLbN1ek93Dy2_hb4BpizO1w |
| linkProvider | Elsevier |
| linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV1NS8NAEB1qPagH0apY68cexJOx7lfSHEUsVWtBbKG3ZbPZQgpNi63iyd_uTj60BfHgNZkl4W2YeWHevAU4N_rahKieorZlPFfxRp7mvu-xQI84pcbKGDu6Tz2_MxAPQzmswG05C4OyyiL35zk9y9bFlWaBZnOWJM0XNBngeAC4yAv5GqwLyQPU9V19_ug8eJi3ltGSAMNL66FM5DV-Nwm6gjKBzU4Wit_L01LJae_AdsEVyU3-OrtQsWkNtpYcBGvQKIKS-YRckJVpj_kePH8r7IjjeQRdIbw48wXBAC-Tk9sPx8PJ1GWOSTGSSZKUoM2le3QycemGYFwul033YdC-6992vOIMBc8wyReeZdTXzESxQ0Ybxyb8MBLSuCIdu30JA6pFpK1k2jETaWJrpQki98NsBdq2cMsPoJpOU3sIRMSOzDEbSz_QgkWOWBrth60oNJQbS2kdLkvo1Cy3ylClhmysMqQVIq1ypOvgl_CqlR1XLpn_tfDovwvPYKPTf-qq7n3vsQGbeCcXphxDdfH6Zk8cvVhEp9nn8wXwaM4N |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Exploring+the+rate-distortion-complexity+optimization+in+neural+image+compression&rft.jtitle=Journal+of+visual+communication+and+image+representation&rft.au=Gao%2C+Yixin&rft.au=Feng%2C+Runsen&rft.au=Guo%2C+Zongyu&rft.au=Chen%2C+Zhibo&rft.date=2024-12-01&rft.issn=1047-3203&rft.volume=105&rft.spage=104294&rft_id=info:doi/10.1016%2Fj.jvcir.2024.104294&rft.externalDBID=n%2Fa&rft.externalDocID=10_1016_j_jvcir_2024_104294 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1047-3203&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1047-3203&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1047-3203&client=summon |