Fine-Tuned Visual Transformer Masked Autoencoder Applied for Anomaly Detection in Satellite Images

• Published in: Applied sciences Vol. 15; no. 11; p. 6286
• Main Authors: Gajda, Jakub, Kwiecień, Joanna
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01.06.2025
• Subjects: Analysis; anomaly detection; autoencoders; Classification; Datasets; Deep learning; Diffusion models; Electric transformers; Machine learning; Medical imaging equipment; Neural networks; satellite images; transformer models
• ISSN: 2076-3417; 2076-3417
• DOI: 10.3390/app15116286

Anomaly detection is a process in which outlier samples can be detected in a given dataset. The purpose of this study is to implement, test, and evaluate the possibility of using deep learning methods for outlier detection with the use of a fine-tuning approach. A Transformer Masked Autoencoder was fine-tuned for a custom satellite image dataset after being pre-trained on the ImageNet subset. The first process of training included building an internal representation of images from a normal class. After adjusting the model weights for this task, a custom dataset with normal and abnormal samples was used for the reconstruction error calculation. The results obtained in this study show that it is possible to distinguish between normal class representatives and outliers using the proposed approach. However, this is not sufficient for the model to be employed in real-life applications. With a given level of precision, the model requires additional knowledge about the subject to correctly classify the sample. To the best of our knowledge, this study is the first to apply ViTMAE for a custom satellite image database. An analysis of the misclassified samples shows that the model tends to generalize the image content and is not sufficiently robust for image noise. As a result of the analysis, a new anomaly indicator is proposed for further study.