CID2013: A Database for Evaluating No-Reference Image Quality Assessment Algorithms

• Published in: IEEE transactions on image processing Vol. 24; no. 1; pp. 390 - 402
• Main Authors: Virtanen, Toni, Nuutinen, Mikko, Vaahteranoksa, Mikko, Oittinen, Pirkko, Hakkinen, Jukka
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.01.2015; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Assessments; Cameras; Categories; Databases, Factual - classification; Databases, Factual - standards; Devices; Distortion; Female; Humans; Image color analysis; Image databases; Image Processing, Computer-Assisted - methods; Image quality; Lighting; Male; no-reference image quality assessment algorithms; Noise; Photography; subjective image quality evaluation; test image databases; test image databases; No-reference image quality assessment algorithms; subjective image quality evaluation
• ISSN: 1057-7149; 1941-0042; 1941-0042
• DOI: 10.1109/TIP.2014.2378061

This paper presents a new database, CID2013, to address the issue of using no-reference (NR) image quality assessment algorithms on images with multiple distortions. Current NR algorithms struggle to handle images with many concurrent distortion types, such as real photographic images captured by different digital cameras. The database consists of six image sets; on average, 30 subjects have evaluated 12-14 devices depicting eight different scenes for a total of 79 different cameras, 480 images, and 188 subjects (67% female). The subjective evaluation method was a hybrid absolute category rating-pair comparison developed for the study and presented in this paper. This method utilizes a slideshow of all images within a scene to allow the test images to work as references to each other. In addition to mean opinion score value, the images are also rated using sharpness, graininess, lightness, and color saturation scales. The CID2013 database contains images used in the experiments with the full subjective data plus extensive background information from the subjects. The database is made freely available for the research community.