Training general representations for remote sensing using in-domain knowledge

• Main Authors: Neumann, Maxim, Pinto, André Susano, Zhai, Xiaohua, Houlsby, Neil
• Format: Journal Article
• Language: English
• Published: 30.09.2020
• Subjects: Computer Science - Computer Vision and Pattern Recognition; Computer Science - Learning
• DOI: 10.48550/arxiv.2010.00332

Automatically finding good and general remote sensing representations allows to perform transfer learning on a wide range of applications - improving the accuracy and reducing the required number of training samples. This paper investigates development of generic remote sensing representations, and explores which characteristics are important for a dataset to be a good source for representation learning. For this analysis, five diverse remote sensing datasets are selected and used for both, disjoint upstream representation learning and downstream model training and evaluation. A common evaluation protocol is used to establish baselines for these datasets that achieve state-of-the-art performance. As the results indicate, especially with a low number of available training samples a significant performance enhancement can be observed when including additionally in-domain data in comparison to training models from scratch or fine-tuning only on ImageNet (up to 11% and 40%, respectively, at 100 training samples). All datasets and pretrained representation models are published online.