Visual Quality Evaluation of Image Object Segmentation: Subjective Assessment and Objective Measure

• Published in: IEEE transactions on image processing Vol. 24; no. 12; pp. 5033 - 5045
• Main Authors: Ran Shi, King Ngi Ngan, Songnan Li, Paramesran, Raveendran, Hongliang Li
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.12.2015; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Databases, Factual - classification; Databases, Factual - standards; Distortion measurement; Humans; Image Processing, Computer-Assisted - methods; Image Processing, Computer-Assisted - standards; Image segmentation; Object segmentation; Objective Measure; Onions; Semantics; Subjective Evaluation; Visual databases; Visual Quality; Visualization; objective measure; visual quality; subjective evaluation; Object segmentation
• ISSN: 1057-7149; 1941-0042
• DOI: 10.1109/TIP.2015.2473099

A visual quality evaluation of image object segmentation as one member of the visual quality evaluation family has been studied over the years. Researchers aim at developing the objective measures that can evaluate the visual quality of object segmentation results in agreement with human quality judgments. It is also significant to construct a platform for evaluating the performance of the objective measures in order to analyze their pros and cons. In this paper, first, we present a novel subjective object segmentation visual quality database, in which a total of 255 segmentation results were evaluated by more than thirty human subjects. Then, we propose a novel full-reference objective measure for an object segmentation visual quality evaluation, which involves four human visual properties. Finally, our measure is compared with some state-of-the-art objective measures on our database. The experiment demonstrates that the proposed measure performs better in matching subjective judgments. Moreover, the database is available publicly for other researchers in the field to evaluate their measures.