Image Compression-Aware Deep Camera ISP Network

• Published in: IEEE access Vol. 9; pp. 137824 - 137832
• Main Authors: Uhm, Kwang-Hyun, Choi, Kyuyeon, Jung, Seung-Won, Ko, Sung-Jea
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Artificial neural networks; Camera ISP; Cameras; compression artifacts; Compression tests; convolutional neural network; Image coding; Image compression; Image quality; JPEG encoders-decoders; Noise reduction; Pipelines; Signal processing; Task analysis; Training; Transform coding
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2021.3116702

Several recent studies have attempted to fully replace the conventional camera image signal processing (ISP) pipeline with convolutional neural networks (CNNs). However, the previous CNN-based ISPs, simply referred to as ISP-Nets, have not explicitly considered that images have to be lossy-compressed in most cases, especially by the off-the-shelf JPEG. To address this issue, in this paper, we propose a novel compression-aware deep camera ISP learning framework. At first, we introduce a new use case of compression artifacts simulation network (CAS-Net), which operates in the opposite way of commonly used compression artifacts reduction networks. Then, the CAS-Net is connected with an ISP-Net such that the ISP network can be trained with consideration of image compression. Throughout experimental studies, we show that our compression-aware camera ISP network can produce images with a better tradeoff between bit-rate and image quality compared to its compression-agnostic version when the performance is evaluated after JPEG compression.