Design and perceptual validation of performance measures for salient object segmentation

Empirical evaluation of salient object segmentation methods requires i) a dataset of ground truth object segmentations and ii) a performance measure to compare the output of the algorithm with the ground truth. In this paper, we provide such a dataset, and evaluate 5 distinct performance measures th...

Full description

Saved in:
Bibliographic Details
Published in2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops pp. 49 - 56
Main Authors Movahedi, Vida, Elder, James H
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.06.2010
Subjects
Algorithm design and analysis
Application software
Computer vision
History
Humans
Image segmentation
Object detection
Object segmentation
Psychology
Shape measurement
Online AccessGet full text

Cover

More Information
Summary:Empirical evaluation of salient object segmentation methods requires i) a dataset of ground truth object segmentations and ii) a performance measure to compare the output of the algorithm with the ground truth. In this paper, we provide such a dataset, and evaluate 5 distinct performance measures that have been used in the literature practically and psychophysically. Our results suggest that a measure based upon minimal contour mappings is most sensitive to shape irregularities and most consistent with human judgements. In fact, the contour mapping measure is as predictive of human judgements as human subjects are of each other. Region-based methods, and contour methods such as Hausdorff distances that do not respect the ordering of points on shape boundaries are significantly less consistent with human judgements. We also show that minimal contour mappings can be used as the correspondence paradigm for Precision-Recall analysis. Our findings can provide guidance in evaluating the results of segmentation algorithms in the future.
ISBN:9781424470297
1424470293
ISSN:2160-7508
DOI:10.1109/CVPRW.2010.5543739