Slimmable Domain Adaptation
Rang Meng, Weijie Chen, Shicai Yang, Jie Song, Luojun Lin, Di Xie,, Shiliang Pu, Xinchao Wang, Mingli Song, Yueting Zhuang

TL;DR
This paper introduces Slimmable Domain Adaptation, a flexible framework enabling models of various sizes to adapt across domains, improving performance on resource-limited devices through a novel distillation and architecture adaptation approach.
Contribution
It proposes a unified framework with a weight-sharing model bank and a new distillation method for effective cross-domain adaptation across different model capacities.
Findings
Outperforms existing methods on multiple benchmarks.
Maintains significant performance even at 1/64 model complexity.
Enables architecture adaptation with an unsupervised evaluation metric.
Abstract
Vanilla unsupervised domain adaptation methods tend to optimize the model with fixed neural architecture, which is not very practical in real-world scenarios since the target data is usually processed by different resource-limited devices. It is therefore of great necessity to facilitate architecture adaptation across various devices. In this paper, we introduce a simple framework, Slimmable Domain Adaptation, to improve cross-domain generalization with a weight-sharing model bank, from which models of different capacities can be sampled to accommodate different accuracy-efficiency trade-offs. The main challenge in this framework lies in simultaneously boosting the adaptation performance of numerous models in the model bank. To tackle this problem, we develop a Stochastic EnsEmble Distillation method to fully exploit the complementary knowledge in the model bank for inter-model…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDomain Adaptation and Few-Shot Learning
