Feed-Forward Latent Domain Adaptation

• Published in: 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) pp. 8475 - 8484
• Main Authors: Bohdal, Ondrej, Li, Da, Hu, Shell Xu, Hospedales, Timothy
• Format: Conference Proceeding
• Language: English
• Published: IEEE 03.01.2024
• Subjects: Adaptation models; Algorithms; and algorithms; Applications; Computer vision; Data models; formulations; Machine learning architectures; Real-time systems; Smartphones / end user devices; Upper bound
• ISSN: 2642-9381
• DOI: 10.1109/WACV57701.2024.00830

We study a new highly-practical problem setting that enables resource-constrained edge devices to adapt a pre-trained model to their local data distributions. Recognizing that device's data are likely to come from multiple latent domains that include a mixture of unlabelled domain-relevant and domain-irrelevant examples, we focus on the comparatively under-studied problem of latent domain adaptation. Considering limitations of edge devices, we aim to only use a pre-trained model and adapt it in a feed-forward way, without using back-propagation and without access to the source data. Modelling these realistic constraints bring us to the novel and practically important problem setting of feed-forward latent domain adaptation. Our solution is to meta-learn a network capable of embedding the mixed-relevance target dataset and dynamically adapting inference for target examples using cross-attention. The resulting framework leads to consistent improvements over strong ERM baselines. We also show that our framework sometimes even improves on the upper bound of domain-supervised adaptation, where only domain-relevant instances are provided for adaptation. This suggests that human annotated domain labels may not always be optimal, and raises the possibility of doing better through automated instance selection.