Logarithmic Lenses: Exploring Log RGB Data for Image Classification

• Published in: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp. 17470 - 17479
• Main Authors: Maxwell, Bruce A., Singhania, Sumegha, Patel, Avnish, Kumar, Rahul, Fryling, Heather, Li, Sihan, Sun, Haonan, He, Ping, Li, Zewen
• Format: Conference Proceeding
• Language: English
• Published: IEEE 16.06.2024
• Subjects: Computer vision; image classification; Image color analysis; physics-based vision; Pipelines; Sensor systems; Training; Transforms; Visual systems
• DOI: 10.1109/CVPR52733.2024.01654

The design of deep network architectures and training methods in computer vision has been well-explored. However, in almost all cases the images have been used as provided, with little exploration of pre-processing steps beyond normalization and data augmentation. Virtually all images posted on the web or captured by devices are processed for viewing by humans. Is the pipeline used for humans also best for use by computers and deep networks? The human visual system uses logarithmic sensors; differences and sums correspond to ratios and products. Features in log space will be invariant to intensity changes and robust to color balance changes. Log RGB space also reveals structure that is corrupted by typical pre-processing. We explore using linear and log RGB data for training standard backbone architectures on an image classification task using data derived directly from RAW images to guarantee its integrity. We found that networks trained on log RGB data exhibit improved performance on an unmodified test set and invariance to intensity and color balance modifications without additional training or data augmentation. Furthermore, we found that the gains from using high quality log data could also be partially or fully realized from data in 8-bit sRGB-JPG format by inverting the sRGB transform and taking the log. These results imply existing databases may benefit from this type of pre-processing. While working with log data, we found it was critical to retain the integrity of the log relationships and that networks using log data train best with meta-parameters different than those used for sRGB or linear data. Finally, we introduce a new 10-category 10k RAW image data set (RAW10) for image classification and other purposes to enable further the exploration of log RGB as an input format for deep networks in computer vision.