An End-to-End Compression Framework Based on Convolutional Neural Networks

• Published in: IEEE transactions on circuits and systems for video technology Vol. 28; no. 10; pp. 3007 - 3018
• Main Authors: Jiang, Feng, Tao, Wen, Liu, Shaohui, Ren, Jie, Guo, Xun, Zhao, Debin
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.10.2018; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Artificial neural networks; Codec; Codecs; Coding standards; compact representation; compression framework; Computer vision; Convolutional codes; convolutional neural networks (CNNs); Decoding; Deep learning; Image coding; Image compression; Image processing; Image quality; Image reconstruction; JPEG encoders-decoders; Machine learning; Neural networks; Noise reduction; Post-processing; Quantization (signal); State of the art; Transform coding
• ISSN: 1051-8215; 1558-2205
• DOI: 10.1109/TCSVT.2017.2734838

Deep learning, e.g., convolutional neural networks (CNNs), has achieved great success in image processing and computer vision especially in high-level vision applications, such as recognition and understanding. However, it is rarely used to solve low-level vision problems such as image compression studied in this paper. Here, we move forward a step and propose a novel compression framework based on CNNs. To achieve high-quality image compression at low bit rates, two CNNs are seamlessly integrated into an end-to-end compression framework. The first CNN, named compact convolutional neural network (ComCNN), learns an optimal compact representation from an input image, which preserves the structural information and is then encoded using an image codec (e.g., JPEG, JPEG2000, or BPG). The second CNN, named reconstruction convolutional neural network (RecCNN), is used to reconstruct the decoded image with high quality in the decoding end. To make two CNNs effectively collaborate, we develop a unified end-to-end learning algorithm to simultaneously learn ComCNN and RecCNN, which facilitates the accurate reconstruction of the decoded image using RecCNN. Such a design also makes the proposed compression framework compatible with existing image coding standards. Experimental results validate that the proposed compression framework greatly outperforms several compression frameworks that use existing image coding standards with the state-of-the-art deblocking or denoising post-processing methods.