Towards user-adaptive remote sensing: Knowledge-driven automatic classification of Sentinel-2 time series

• Published in: Remote sensing of environment Vol. 264; p. 112615
• Main Authors: Arvor, Damien, Betbeder, Julie, Daher, Felipe R.G., Blossier, Tim, Le Roux, Renan, Corgne, Samuel, Corpetti, Thomas, de Freitas Silgueiro, Vinicius, Silva Junior, Carlos Antonio da
• Format: Journal Article
• Language: English
• Published: New York Elsevier Inc 01.10.2021; Elsevier BV; Elsevier
• Subjects: Amazon; Classification; Continental interfaces, environment; End users; Engineering Sciences; Environment and Society; Environmental Engineering; Environmental Sciences; Fluids mechanics; Forest watersheds; Heterogeneity; Hybrid systems; Image analysis; Image classification; Image processing; Information processing; Knowledge representation; Knowledge-driven; Land cover; Land use; Mechanics; Natural vegetation; NGOs; Nongovernmental organizations; Ontologies; Remote sensing; Riparian forests; Satellite data; Satellites; Sciences of the Universe; Sentinel-2; Signal and Image processing; Spatial heterogeneity; Spectral signatures; Time series; User requirements; Vegetation; Ontologies; Land cover; Amazon; Knowledge-driven; Sentinel-2; Time series
• ISSN: 0034-4257; 1879-0704
• DOI: 10.1016/j.rse.2021.112615

Land cover mapping over large areas is essential to address a wide spectrum of socio-environmental challenges. For this reason, many global or regional land cover products are regularly released to the scientific community. Yet, the remote sensing community has not fully addressed the challenge to extract useful information from vast volumes of satellite data. Especially, major limitations concern the use of inadequate classification schemes and “black box” methods that may not match with end-users conceptualization of geographic features. In this paper, we introduce a knowledge-driven methodological approach to automatically process Sentinel-2 time series in order to produce pre-classifications that can be adapted by end-users to match their requirements. The approach relies on a conceptual framework inspired from ontologies of scientific observation and geographic information to describe the representation of geographic entities in remote sensing images. The implementation consists in a three-stage classification system including an initial stage, a dichotomous stage and a modular stage. At each stage, the system firstly relies on natural language semantic descriptions of time series of spectral signatures before assigning labels of land cover classes. The implementation was tested on 75 time series of Sentinel-2 images (i.e. 2069 images) in the Southern Brazilian Amazon to map natural vegetation and water bodies as required by a local end-user, i.e. a non-governmental organization. The results confirmed the potential of the method to accurately detect water bodies (F-score = 0.874 for bodies larger than 10 m) and map natural vegetation (max F-score = 0.875), yet emphasizing the spatial heterogeneity of accuracy results. In addition, it proved to be efficient to provide rapid estimates of degraded riparian forests at watershed level (R2 = 0.871). Finally, we discuss potential improvements both in the system's implementation, e.g. considering additional characteristics, and in the conceptual framework, e.g. moving from pixel- to object-based image analysis and evolving towards a hybrid system combining data- and knowledge-driven approaches. •End-user requirements are rarely taken into account in large scale land cover maps.•We developed a knowlegde-driven approach to automatically map large areas.•We introduced an ontology-based conceptual framework.•We implemented a system to classify Sentinel 2 times series in the Brazilian Amazon.•The system was efficient to map vegetation gradients and small water bodies.