Material based salient object detection from hyperspectral images

• Published in: Pattern recognition Vol. 76; pp. 476 - 490
• Main Authors: Liang, Jie, Zhou, Jun, Tong, Lei, Bai, Xiao, Wang, Bin
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.04.2018
• Subjects: Hyperspectral imaging; Hyperspectral unmixing; Material composition; Salient object detection; Spectral-spatial distribution; Hyperspectral unmixing; Material composition; Spectral-spatial distribution; Salient object detection; Hyperspectral imaging
• ISSN: 0031-3203; 1873-5142
• DOI: 10.1016/j.patcog.2017.11.024

•To the best of our knowledge, this is the first time that salient objects are detected based on extracting explicit material property embedded in the spectral responses via retrieval of endmembers and estimating their abundance.•The novelty also comes from adopting hyperspectral unmixing model as a preprocessing step for salient object detection. This allows the spatial distribution of endmembers be estimated, so the method is capable of dealing with mixed spectral responses in low spatial resolution hyperspectral images.•Different from existing hyperspectral salient object detection methods, we developed a novel method to fuse both local and global features for hyperspectral salient object detection.•We built a hyperspectral image dataset for salient detection, which contains mixed objects with similar color but different materials. While salient object detection has been studied intensively by the computer vision and pattern recognition community, there are still great challenges in practical applications, especially when perceived objects have similar appearance such as intensity, color, and orientation, but different materials. Traditional methods do not provide good solution to this problem since they were mostly developed on color images and do not have the full capability in discriminating materials. More advanced technology and methodology are in demand to gain access to further information beyond human vision. In this paper, we extend the concept of salient object detection to material level based on hyperspectral imaging and present a material-based salient object detection method which can effectively distinguish objects with similar perceived color but different spectral responses. The proposed method first estimates the spatial distribution of different materials or endmembers using a hyperspectral unmixing approach. This step enables the calculation of a conspicuity map based on the global spatial variance of spectral responses. Then the multi-scale center-surround difference of local spectral features is calculated via spectral distance measures to generate local spectral conspicuity maps. These two types of conspicuity maps are fused for the final salient object detection. A new dataset of 45 hyperspectral images is introduced for experimental validation. The results show that our method outperforms several existing hyperspectral salient object detection approaches and the state-of-the-art methods proposed for RGB images.