Evaluating image quality measures to assess the impact of lossy data compression applied to climate simulation data

• Published in: Computer graphics forum Vol. 38; no. 3; pp. 517 - 528
• Main Authors: Baker, A. H., Hammerling, D. M., Turton, T. L.
• Format: Journal Article Conference Proceeding
• Language: English
• Published: Oxford Blackwell Publishing Ltd 01.06.2019
• Subjects: Categories and Subject Descriptors (according to ACM CCS); Climate models; Compression tests; Computer simulation; Data compression; data compression, scientific visualization, climate science, visual evaluation; E.4 [Coding and Information Theory]: Data Compaction and Compression; H.1.2 [User/Machine Systems]: Human factors; I.5.2 [Design Methodology]: Feature evaluation; Image compression; Image quality; Quality assessment; Scientific visualization; Software reviews; Statistical models
• ISSN: 0167-7055; 1467-8659
• DOI: 10.1111/cgf.13707

Applying lossy data compression to climate model output is an attractive means of reducing the enormous volumes of data generated by climate models. However, because lossy data compression does not exactly preserve the original data, its application to scientific data must be done judiciously. To this end, a collection of measures is being developed to evaluate various aspects of lossy compression quality on climate model output. Given the importance of data visualization to climate scientists interacting with model output, any suite of measures must include a means of assessing whether images generated from the compressed model data are noticeably different from images based on the original model data. Therefore, in this work we conduct a forced‐choice visual evaluation study with climate model data that surveyed more than one hundred participants with domain relevant expertise. In addition to the images created from unaltered climate model data, study images are generated from model data that is subjected to two different types of lossy compression approaches and multiple levels (amounts) of compression. Study participants indicate whether a visual difference can be seen, with respect to the reference image, due to lossy compression effects. We assess the relationship between the perceptual scores from the user study to a number of common (full reference) image quality assessment (IQA) measures, and use statistical models to suggest appropriate measures and thresholds for evaluating lossily compressed climate data. We find the structural similarity index (SSIM) to perform the best, and our findings indicate that the threshold required for climate model data is much higher than previous findings in the literature.