Efficient Source-Free Time-Series Adaptation via Parameter Subspace Disentanglement
In this paper, we propose a framework for efficient Source-Free Domain Adaptation (SFDA) in the context of time-series, focusing on enhancing both parameter efficiency and data-sample utilization. Our approach introduces an improved paradigm for source-model preparation and target-side adaptation, a...
Saved in:
Main Authors | , , , , , , |
---|---|
Format | Journal Article |
Language | English |
Published |
02.10.2024
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | In this paper, we propose a framework for efficient Source-Free Domain
Adaptation (SFDA) in the context of time-series, focusing on enhancing both
parameter efficiency and data-sample utilization. Our approach introduces an
improved paradigm for source-model preparation and target-side adaptation,
aiming to enhance training efficiency during target adaptation. Specifically,
we reparameterize the source model's weights in a Tucker-style decomposed
manner, factorizing the model into a compact form during the source model
preparation phase. During target-side adaptation, only a subset of these
decomposed factors is fine-tuned, leading to significant improvements in
training efficiency. We demonstrate using PAC Bayesian analysis that this
selective fine-tuning strategy implicitly regularizes the adaptation process by
constraining the model's learning capacity. Furthermore, this
re-parameterization reduces the overall model size and enhances inference
efficiency, making the approach particularly well suited for
resource-constrained devices. Additionally, we demonstrate that our framework
is compatible with various SFDA methods and achieves significant computational
efficiency, reducing the number of fine-tuned parameters and inference overhead
in terms of MACs by over 90% while maintaining model performance. |
---|---|
DOI: | 10.48550/arxiv.2410.02147 |