Spectral-Spatial Feature Extraction Network With SSM-CNN for Hyperspectral-Multispectral Image Collaborative Classification

Multisource remote sensing (RS) image classification is a significant research area in Earth observation, aiming to achieve more comprehensive and accurate classification of land cover by integrating data from different sensors. Due to differences in imaging mechanisms and information imbalance betw...

Full description

Saved in:
Bibliographic Details
Published inIEEE journal of selected topics in applied earth observations and remote sensing Vol. 17; pp. 17555 - 17566
Main Authors Wang, Qingwang, Fan, Xingxing, Huang, Jiangbo, Li, Shuai, Shen, Tao
Format Journal Article
LanguageEnglish
Published Piscataway IEEE 2024
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects
Online AccessGet full text

Cover

Loading…
Abstract Multisource remote sensing (RS) image classification is a significant research area in Earth observation, aiming to achieve more comprehensive and accurate classification of land cover by integrating data from different sensors. Due to differences in imaging mechanisms and information imbalance between multisource data, multisource RS image classification faces two major challenges as follows. 1) Synergistically capturing features from different modalities to fully exploit complementary information. 2) Adaptively fusing multisource features to overcome the imbalance between modalities and avoid redundant information. This article proposes a spectral-spatial feature extraction network with SSM-CNN (SSFNet) for the collaborative classification of hyperspectral images (HSI) and multispectral images (MSI). Specifically, SSFNet captures long-range spectral correlations in HSI through a bidirectional state-space model (SSM) and learns local correlations between adjacent channels through spectral grouping, achieving global-local spectral information mining in HSI. Simultaneously, joint spatial feature extraction for HSI and MSI data is performed using embedded weight-shared residual feature extractor based on convolutional neural network. This process involves adaptively identifying the importance of features through privatized factors in batch normalization and accurately replacing redundant features. In addition, a spatial attention module is used to further enhance spatial feature representation. Finally, to better accommodate feature distributions and enhance classification outcomes, the extracted spectral-spatial features are combined using weighted fusion, allowing for dynamic integration. Experimental results on two datasets demonstrate that the proposed SSFNet significantly outperforms other competing methods.
AbstractList Multisource remote sensing (RS) image classification is a significant research area in Earth observation, aiming to achieve more comprehensive and accurate classification of land cover by integrating data from different sensors. Due to differences in imaging mechanisms and information imbalance between multisource data, multisource RS image classification faces two major challenges as follows. 1) Synergistically capturing features from different modalities to fully exploit complementary information. 2) Adaptively fusing multisource features to overcome the imbalance between modalities and avoid redundant information. This article proposes a spectral-spatial feature extraction network with SSM-CNN (SSFNet) for the collaborative classification of hyperspectral images (HSI) and multispectral images (MSI). Specifically, SSFNet captures long-range spectral correlations in HSI through a bidirectional state-space model (SSM) and learns local correlations between adjacent channels through spectral grouping, achieving global-local spectral information mining in HSI. Simultaneously, joint spatial feature extraction for HSI and MSI data is performed using embedded weight-shared residual feature extractor based on convolutional neural network. This process involves adaptively identifying the importance of features through privatized factors in batch normalization and accurately replacing redundant features. In addition, a spatial attention module is used to further enhance spatial feature representation. Finally, to better accommodate feature distributions and enhance classification outcomes, the extracted spectral-spatial features are combined using weighted fusion, allowing for dynamic integration. Experimental results on two datasets demonstrate that the proposed SSFNet significantly outperforms other competing methods.
Author Shen, Tao
Fan, Xingxing
Wang, Qingwang
Li, Shuai
Huang, Jiangbo
Author_xml – sequence: 1
  givenname: Qingwang
  orcidid: 0000-0001-5820-5357
  surname: Wang
  fullname: Wang, Qingwang
  email: wangqingwang@kust.edu.cn
  organization: Faculty of Information Engineering and Automation, Kunming, China
– sequence: 2
  givenname: Xingxing
  surname: Fan
  fullname: Fan, Xingxing
  email: 20222104015@stu.kust.edu.cn
  organization: Faculty of Information Engineering and Automation, Kunming, China
– sequence: 3
  givenname: Jiangbo
  orcidid: 0000-0002-6103-7769
  surname: Huang
  fullname: Huang, Jiangbo
  email: huangjiangbo@stu.kust.edu.cn
  organization: Faculty of Information Engineering and Automation, Kunming, China
– sequence: 4
  givenname: Shuai
  orcidid: 0000-0002-4260-7949
  surname: Li
  fullname: Li, Shuai
  email: lishuai@kust.edu.cn
  organization: School of Architectural Engineering, Kunming University of Science and Technology, Kunming, China
– sequence: 5
  givenname: Tao
  orcidid: 0000-0003-1273-7950
  surname: Shen
  fullname: Shen, Tao
  email: shentao@kust.edu.cn
  organization: Faculty of Information Engineering and Automation, Kunming, China
BookMark eNpNkU1v1DAQhi3USmxbfgEcInHOYscfcY7Vqh-L2kVqKvVoTZxx8ZKug-MFKv48XtIiLjOa0bzPjOY9IUe7sENC3jO6ZIw2nz639-d37bKilVhyoYTS7A1ZVEyykkkuj8iCNbwpmaDiLTmZpi2lqqobviC_2xFtijCU7QjJw1BcIqR9xOLiV27b5MOu2GD6GeK34sGnr0Xb3parzaZwIRbXzyPG6ZVwux-Sf62K9RM8YrEKwwBdiJn9I1cDTJN33sKBe0aOHQwTvnvJp6S9vLhfXZc3X67Wq_Ob0la6SaXTDBqB1uUgalV1SmInGYKSoBnWtda818x10vV9pSgVQgMHKrqOMuCnZD1T-wBbM0b_BPHZBPDmbyPERwMxeTug4bUEq3rXs_zIyiHQvqEooVYWaEdVZn2cWWMM3_c4JbMN-7jLxxvOmJR1JZomT_F5ysYwTRHdv62MmoNfZvbLHPwyL35l1YdZ5RHxP4XSIoP5H4Bylas
CODEN IJSTHZ
Cites_doi 10.1109/TGRS.2014.2317499
10.1109/JSTARS.2019.2962659
10.1049/iet-ipr.2016.0421
10.1007/s11431-023-2528-8
10.1109/TGRS.2016.2530807
10.1109/TNNLS.2022.3189994
10.1109/TGRS.2024.3430985
10.1109/TGRS.2020.2969024
10.1109/TGRS.2023.3344698
10.1109/TGRS.2021.3130716
10.1109/JSTARS.2015.2432037
10.1109/LGRS.2017.2704625
10.1109/TGRS.2024.3423759
10.1109/ICICML60161.2023.10424918
10.1109/JSTARS.2024.3378348
10.1109/TNNLS.2020.3028945
10.1109/TNNLS.2022.3171572
10.1109/JSTARS.2016.2634863
10.1109/TGRS.2015.2421051
10.1109/TGRS.2023.3286826
10.1109/JSTARS.2024.3439560
10.1109/TGRS.2017.2756851
10.1109/TGRS.2023.3284671
10.1109/TGRS.2020.3016820
10.1109/JSTARS.2024.3403863
10.1109/TGRS.2022.3169216
10.1109/JSTARS.2022.3232995
10.1016/j.isprsjprs.2021.05.011
10.1109/JSTARS.2020.3040305
10.1109/TGRS.2024.3351846
10.1109/LGRS.2014.2350263
10.1109/TGRS.2021.3097093
10.1109/LGRS.2024.3407111
10.1109/TNNLS.2022.3149394
10.1109/tgrs.2022.3231930
10.1109/TGRS.2023.3311535
ContentType Journal Article
Copyright Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024
Copyright_xml – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024
DBID 97E
ESBDL
RIA
RIE
AAYXX
CITATION
7UA
8FD
C1K
F1W
FR3
H8D
H96
KR7
L.G
L7M
DOA
DOI 10.1109/JSTARS.2024.3464681
DatabaseName IEEE All-Society Periodicals Package (ASPP) 2005-present
IEEE Xplore Open Access Journals
IEEE All-Society Periodicals Package (ASPP) 1998-Present
IEEE Electronic Library Online
CrossRef
Water Resources Abstracts
Technology Research Database
Environmental Sciences and Pollution Management
ASFA: Aquatic Sciences and Fisheries Abstracts
Engineering Research Database
Aerospace Database
Aquatic Science & Fisheries Abstracts (ASFA) 2: Ocean Technology, Policy & Non-Living Resources
Civil Engineering Abstracts
Aquatic Science & Fisheries Abstracts (ASFA) Professional
Advanced Technologies Database with Aerospace
Directory of Open Access Journals
DatabaseTitle CrossRef
Aerospace Database
Civil Engineering Abstracts
Aquatic Science & Fisheries Abstracts (ASFA) Professional
Aquatic Science & Fisheries Abstracts (ASFA) 2: Ocean Technology, Policy & Non-Living Resources
Technology Research Database
ASFA: Aquatic Sciences and Fisheries Abstracts
Engineering Research Database
Advanced Technologies Database with Aerospace
Water Resources Abstracts
Environmental Sciences and Pollution Management
DatabaseTitleList

Aerospace Database
Database_xml – sequence: 1
  dbid: DOA
  name: DOAJ Directory of Open Access Journals
  url: https://www.doaj.org/
  sourceTypes: Open Website
– sequence: 2
  dbid: RIE
  name: IEEE Electronic Library Online
  url: https://proxy.k.utb.cz/login?url=https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Geology
EISSN 2151-1535
EndPage 17566
ExternalDocumentID oai_doaj_org_article_375ac6dfd10242fea0d90e5a76ca0b06
10_1109_JSTARS_2024_3464681
10684557
Genre orig-research
GrantInformation_xml – fundername: National Natural Science Foundation of China
  grantid: 62201237
  funderid: 10.13039/501100001809
– fundername: Major Science and Technology Projects in Yunnan Province
  grantid: 202302AG050009
  funderid: 10.13039/501100018531
– fundername: Yunnan Fundamental Research Projects
  grantid: 202101BE070001-008; 202401AW070019; 202301AV070003
GroupedDBID 0R~
29I
4.4
5GY
5VS
6IK
97E
AAFWJ
AAJGR
ACIWK
AENEX
AETIX
AFRAH
ALMA_UNASSIGNED_HOLDINGS
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
DU5
EBS
EJD
ESBDL
GROUPED_DOAJ
HZ~
IFIPE
IPLJI
JAVBF
M43
O9-
OCL
OK1
RIA
RIE
RIG
RNS
AAYXX
AGSQL
CITATION
7UA
8FD
C1K
F1W
FR3
H8D
H96
KR7
L.G
L7M
ID FETCH-LOGICAL-c289t-f81a94ecf94e4762b65eb51ea65a81e77883d81fb5fdd2600448a3a04bb01a3
IEDL.DBID DOA
ISSN 1939-1404
IngestDate Mon Oct 21 19:39:31 EDT 2024
Thu Nov 07 15:28:56 EST 2024
Fri Dec 06 07:14:53 EST 2024
Wed Oct 23 05:52:14 EDT 2024
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Language English
License https://creativecommons.org/licenses/by-nc-nd/4.0
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c289t-f81a94ecf94e4762b65eb51ea65a81e77883d81fb5fdd2600448a3a04bb01a3
ORCID 0000-0003-1273-7950
0000-0002-4260-7949
0000-0001-5820-5357
0000-0002-6103-7769
OpenAccessLink https://doaj.org/article/375ac6dfd10242fea0d90e5a76ca0b06
PQID 3115572499
PQPubID 75722
PageCount 12
ParticipantIDs crossref_primary_10_1109_JSTARS_2024_3464681
proquest_journals_3115572499
doaj_primary_oai_doaj_org_article_375ac6dfd10242fea0d90e5a76ca0b06
ieee_primary_10684557
PublicationCentury 2000
PublicationDate 20240000
2024-00-00
20240101
2024-01-01
PublicationDateYYYYMMDD 2024-01-01
PublicationDate_xml – year: 2024
  text: 20240000
PublicationDecade 2020
PublicationPlace Piscataway
PublicationPlace_xml – name: Piscataway
PublicationTitle IEEE journal of selected topics in applied earth observations and remote sensing
PublicationTitleAbbrev JSTARS
PublicationYear 2024
Publisher IEEE
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml – name: IEEE
– name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References ref13
ref35
ref12
ref34
ref15
ref14
ref36
ref31
ref30
ref11
ref33
ref10
ref32
ref2
ref1
ref17
ref16
ref19
ref18
ref24
ref23
ref26
ref25
ref20
ref22
ref21
ref28
ref27
ref29
ref8
ref7
ref9
ref4
ref3
ref6
ref5
References_xml – ident: ref14
  doi: 10.1109/TGRS.2014.2317499
– ident: ref11
  doi: 10.1109/JSTARS.2019.2962659
– ident: ref12
  doi: 10.1049/iet-ipr.2016.0421
– ident: ref35
  doi: 10.1007/s11431-023-2528-8
– ident: ref6
  doi: 10.1109/TGRS.2016.2530807
– ident: ref24
  doi: 10.1109/TNNLS.2022.3189994
– ident: ref26
  doi: 10.1109/TGRS.2024.3430985
– ident: ref31
  doi: 10.1109/TGRS.2020.2969024
– ident: ref5
  doi: 10.1109/TGRS.2023.3344698
– ident: ref28
  doi: 10.1109/TGRS.2021.3130716
– ident: ref13
  doi: 10.1109/JSTARS.2015.2432037
– ident: ref19
  doi: 10.1109/LGRS.2017.2704625
– ident: ref36
  doi: 10.1109/TGRS.2024.3423759
– ident: ref25
  doi: 10.1109/ICICML60161.2023.10424918
– ident: ref1
  doi: 10.1109/JSTARS.2024.3378348
– ident: ref3
  doi: 10.1109/TNNLS.2020.3028945
– ident: ref10
  doi: 10.1109/TNNLS.2022.3171572
– ident: ref18
  doi: 10.1109/JSTARS.2016.2634863
– ident: ref16
  doi: 10.1109/TGRS.2015.2421051
– ident: ref23
  doi: 10.1109/TGRS.2023.3286826
– ident: ref8
  doi: 10.1109/JSTARS.2024.3439560
– ident: ref20
  doi: 10.1109/TGRS.2017.2756851
– ident: ref17
  doi: 10.1109/TGRS.2023.3284671
– ident: ref4
  doi: 10.1109/TGRS.2020.3016820
– ident: ref7
  doi: 10.1109/JSTARS.2024.3403863
– ident: ref9
  doi: 10.1109/TGRS.2022.3169216
– ident: ref22
  doi: 10.1109/JSTARS.2022.3232995
– ident: ref30
  doi: 10.1016/j.isprsjprs.2021.05.011
– ident: ref21
  doi: 10.1109/JSTARS.2020.3040305
– ident: ref2
  doi: 10.1109/TGRS.2024.3351846
– ident: ref15
  doi: 10.1109/LGRS.2014.2350263
– ident: ref32
  doi: 10.1109/TGRS.2021.3097093
– ident: ref27
  doi: 10.1109/LGRS.2024.3407111
– ident: ref29
  doi: 10.1109/TNNLS.2022.3149394
– ident: ref33
  doi: 10.1109/tgrs.2022.3231930
– ident: ref34
  doi: 10.1109/TGRS.2023.3311535
SSID ssj0062793
Score 2.392211
Snippet Multisource remote sensing (RS) image classification is a significant research area in Earth observation, aiming to achieve more comprehensive and accurate...
SourceID doaj
proquest
crossref
ieee
SourceType Open Website
Aggregation Database
Publisher
StartPage 17555
SubjectTerms Artificial neural networks
Classification
Collaboration
collaborative classification
Convolutional neural network (CNN)
Convolutional neural networks
Correlation
Data mining
feature exchange
Feature extraction
Hyperspectral imaging
Image classification
Land cover
Laser radar
multisource remote sensing (RS)
Neural networks
Remote sensing
Spatial data
state–space model (SSM)
Transformers
SummonAdditionalLinks – databaseName: IEEE Electronic Library Online
  dbid: RIE
  link: http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1Lb9QwELZoJSQulEcRWwrygSNe7PUrPpZVy4LUHFgQvVm24wiE2KIli3j8eWbshKeQuERx5MROZmzP58x8Q8hDmZ3JvVownpJjKsrIwI5TLLk-SCFNzIXs-bw1q1fq-YW-GIPVSyxMzrk4n-U5npZ_-d1l2uFWGYxw0yit7R7Zs87UYK1p2jULWxh2wSBxDDljRoohwd1j0PGTF2sAgws1l8oo04jflqHC1j-mV_lrTi4LzdkBaacuVv-Sd_PdEOfp6x_sjf_9DjfI9dHkpCdVR26SK3lzi1x9WlL6frlNvmEKetzvYJieGNSRolm422Z6-nnY1rgH2lZvcfr67fCGrtfnbNm2FOxdugIcW8M18QklnHcq0WfvYbKiy5-a9glKaK2je1LRiEOyPjt9uVyxMSUDS4DMBtY3IjiVUw8HBfNoNDpHLXIwOjQiWwDUsmtEH3Xfdch9D-gvyMBVjFwEeYfsby43-S6h0sLyyTvAJ9bA80ITVWoMVEkhIkSdkUeTfPyHyrvhC17hzldxehSnH8U5I09Qhj-qIml2uQDf3o9j0EurQzJd3wk0TPoceOd41sGaFHjk0OYhyuuX9qqoZuR4Ugk_jvCPHlmKtAXw6o7-cds9cg27WPdrjsn-sN3l-2DBDPFB0dzvkpnt0Q
  priority: 102
  providerName: IEEE
Title Spectral-Spatial Feature Extraction Network With SSM-CNN for Hyperspectral-Multispectral Image Collaborative Classification
URI https://ieeexplore.ieee.org/document/10684557
https://www.proquest.com/docview/3115572499
https://doaj.org/article/375ac6dfd10242fea0d90e5a76ca0b06
Volume 17
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV3PSxwxFA4iFHoRbRXXWsmhR6PJ5sdMjnbRroJ76Cp6C0kmgx5cZZ0tlv7zfS-ZbRUPXrwMZAjJ5OXNe-8Led8j5JtM1qRWDRmP0TIVZGAQxykWbeulkCakTPZ8PjHjS3V2ra-flfrCO2GFHrgI7lBW2kfTtI1Ab9ImzxvLk_aViZ6HnmybD5dgqthgM6wy3S5EJ5YhgUzPNyS4PQSFP_o5BWQ4VAdSGWVq8cInZer-vtbKKwOdvc7JOlnrw0V6VD5zg6yk2Sfy4Ucux_v7M_mD5ePxrIJhaWFQJYoh3WKe6PFTNy85C3RSbnrTq9vuhk6n52w0mVCIVekYMGhJtcQRciruskVP78DQ0NF_LfkFLYy08WpR3s1NMj05vhiNWV9OgUVAVR1ra-GtSrGFhwIbGIxOQYvkjfa1SBWAYdnUog26bRrkrQfk5qXnKgQuvNwiq7P7WdomVFbg-ngD2KIyMJ6vg4q1gS7RB4SXA7K_FKd7KJwZLmMNbl2RvkPpu176A_IdRf6vKxJe5xegBq5XA_eWGgzIJm7Ys_lMrbSuBmR3uYOu_zsfHTIM6QqAp915j7m_kI-4nnIws0tWu_kifYVQpQt7WSv3clbhX04e5Vw
link.rule.ids 314,780,784,796,864,2102,4024,27923,27924,27925,54758
linkProvider Directory of Open Access Journals
linkToHtml http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1Lb9QwELagCMGFZxELBXzgiBd7_Uh8LKuWLXRzYIvozbIdR0WILVqyiMefZ8ZOeAqJSxRHznPG9vc5nm8IeSyTNalTM8ZjtEwFGRjgOMWi7bwU0oSUxZ6XjVm8Vi9O9ekQrJ5jYVJKefFZmuJu_pffnsctTpVBCze10rq6SC5pBUC3hGuNHa-ZVVljFyCJZagaM4gMCW6fgpfvv1oBHZypqVRGmVr8NhBlvf4hwcpfvXIeag6vk2Z8yLLC5N1024dp_PqHfuN_v8UNcm0AnXS_eMlNciGtb5HLz3NS3y-3yTdMQo8zHgwTFINDUgSG202iB5_7TYl8oE1ZL07fvO3P6Gq1ZPOmoYB46QKYbAnYxCvkgN6xRI_eQ3dF5z997ROUEK_jAqXsE7tkdXhwMl-wISkDi8DNetbVwluVYgcbBT1pMDoFLZI32tciVUCpZVuLLuiubVH9Hvifl56rELjw8g7ZWZ-v011CZQUDKG-BoVQGrufroGJtoEr0AUnqhDwZ7eM-FOUNlxkLt66Y06E53WDOCXmGNvxRFWWz8wH49m5ohU5W2kfTdq1AaNIlz1vLk_aViZ4HDvfcRXv9cr9iqgnZG13CDW38o0OdIl0BfbX3_nHaI3JlcbI8dsdHzcv75Co-bpm92SM7_WabHgCe6cPD7MXfATo98SQ
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Spectral%E2%80%93Spatial+Feature+Extraction+Network+With+SSM%E2%80%93CNN+for+Hyperspectral%E2%80%93Multispectral+Image+Collaborative+Classification&rft.jtitle=IEEE+journal+of+selected+topics+in+applied+earth+observations+and+remote+sensing&rft.au=Wang%2C+Qingwang&rft.au=Fan%2C+Xingxing&rft.au=Huang%2C+Jiangbo&rft.au=Li%2C+Shuai&rft.date=2024&rft.issn=1939-1404&rft.eissn=2151-1535&rft.volume=17&rft.spage=17555&rft.epage=17566&rft_id=info:doi/10.1109%2FJSTARS.2024.3464681&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_JSTARS_2024_3464681
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1939-1404&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1939-1404&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1939-1404&client=summon