Effective ISP Tuning Framework Based on User Preference Feedback

• Published in: Electronic Imaging Vol. 32; no. 9; pp. 316-1 - 316-5
• Main Authors: Yang, Cheoljong, Kim, Jinhyun, Lee, Jungmin, Kim, Younghoon, Kim, Sung-Su, Kim, TaeHyung, Yim, JoonSeo
• Format: Journal Article
• Language: English
• Published: 7003 Kilworth Lane, Springfield, VA 22151 USA Society for Imaging Science and Technology 26.01.2020
• Subjects: CMOS image sensor; Deep Neural Network; Fine-Tuning; Image Quality Metric; Image Signal Processor; Perceptual Metric; User Feedback
• ISSN: 2470-1173; 2470-1173
• DOI: 10.2352/ISSN.2470-1173.2020.9.IQSP-316

This paper presents an effective tuning framework between CMOS Image Sensor (CIS) and Image Signal Processor (ISP) based on user preference feedback. One of key issue in ISP tuning is how to apply individual's subjectivity of Image Quality (IQ) in systematic way. In order to mitigate this issue, we propose a framework that efficiently surveys user preference of IQ and select ISP parameter based on those preferences. The overall processes are done on large-scale image database generated by an ISP simulator. In preference survey part, we make clusters that consist of perceptually similar images and gather user's feedback on representative images of each cluster. Next, for training user preference, we train a DNN model according to general preference, and fine-tune model to optimize individuals preference based on user feedback. The model provides ISP candidate most similar to the preferences. In order to assess performance, the proposed framework was evaluated with a state-of-art CIS and ISP system. The experimental results indicate that the proposed framework converges the IQ score according to user feedback and find the ISP parameters that have higher quality IQ as compared with hand-tuned results.