Image Co-Saliency Detection and Co-Segmentation via Progressive Joint Optimization

• Published in: IEEE transactions on image processing Vol. 28; no. 1; pp. 56 - 71
• Main Authors: Tsai, Chung-Chi, Li, Weizhi, Hsu, Kuang-Jui, Qian, Xiaoning, Lin, Yen-Yu
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.01.2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Co-saliency detection; co-segmentation; Computational modeling; Energy conservation; energy minimization; Image detection; Image segmentation; joint optimization; locally adaptive proposal fusion; Optimization; Proposals; Salience; Saliency detection; Task analysis; Visualization
• ISSN: 1057-7149; 1941-0042
• DOI: 10.1109/TIP.2018.2861217

We present a novel computational model for simultaneous image co-saliency detection and co-segmentation that concurrently explores the concepts of saliency and objectness in multiple images. It has been shown that the co-saliency detection via aggregating multiple saliency proposals by diverse visual cues can better highlight the salient objects; however, the optimal proposals are typically region-dependent and the fusion process often leads to blurred results. Co-segmentation can help preserve object boundaries, but it may suffer from complex scenes. To address these issues, we develop a unified method that addresses co-saliency detection and co-segmentation jointly via solving an energy minimization problem over a graph. Our method iteratively carries out the region-wise adaptive saliency map fusion and object segmentation to transfer useful information between the two complementary tasks. Through the optimization iterations, sharp saliency maps are gradually obtained to recover entire salient objects by referring to object segmentation, while these segmentations are progressively improved owing to the better saliency prior. We evaluate our method on four public benchmark data sets while comparing it to the state-of-the-art methods. Extensive experiments demonstrate that our method can provide consistently higher-quality results on both co-saliency detection and co-segmentation.