Camera Image Quality Tradeoff Processing of Image Sensor Re-mosaic using Deep Neural Network

• Published in: Electronic Imaging Vol. 33; no. 9; pp. 206-1 - 206-7
• Main Authors: Kim, Younghoon, Lee, Jungmin, Kim, SungSu, Bang, Jiyun, Hong, Dagyum, Kim, TaeHyung, Yim, JoonSeo
• Format: Journal Article
• Language: English
• Published: IS&T 7003 Kilworth Lane Springfield, VA 22151 USA Society for Imaging Science and Technology 18.01.2021
• Subjects: Deep Neural Network; Image Quality Metrics; Image Sensor; Image Signal Processor
• ISSN: 2470-1173; 2470-1173
• DOI: 10.2352/ISSN.2470-1173.2021.9.IQSP-206

Recently, with the release of 108 mega pixel resolution image sensor, the photo quality of smartphone camera, including detail, and texture, is getting much higher. This became possible only because by utilizing the remosaic technology which re-organize color filter arrays into the Bayer patterns compatible to existing Image Signal Processor (ISP) of commodity AP. However, the optimized parameter configurations of the remosaic block require lots of efforts and long tuning period in order to secure the desired image quality level and sensor characteristics. This paper proposes a deep neural network based camera auto-tuning system for the remosaic ISP block. Firstly, considering the learning phase, big image quality database is created in the random way using reference image and tuning register. Second, the virtual ISP model has been trained in order that predicts image quality by changing sensor tuning registers. Finally, the optimization layer generates the sensor remosaic parameters in order to achieve the user's target image quality expectation. By experiment, the proposed system has been verified to secure the image quality at the level of professionally hand-tuned photography. Especially, the remosaic artifact of false color, color desaturation and line broken artifacts are improved significantly by more than 23%, 4%, and 12%, respectively.