Continual Domain Adaptation through Pruning-aided Domain-specific Weight Modulation

TL;DR

This paper introduces a continual domain adaptation method that leverages pruning and domain-specific weight modulation to adapt models to changing domains while preventing forgetting of previous domains.

Contribution

It presents a novel pruning-based framework for preserving domain-specific features and a batch-norm based metric for effective inference in continual unsupervised domain adaptation.

Findings

Achieves state-of-the-art performance in continual domain adaptation

Effectively prevents catastrophic forgetting of past domains

Utilizes pruning to preserve domain-specific knowledge

Abstract

In this paper, we propose to develop a method to address unsupervised domain adaptation (UDA) in a practical setting of continual learning (CL). The goal is to update the model on continually changing domains while preserving domain-specific knowledge to prevent catastrophic forgetting of past-seen domains. To this end, we build a framework for preserving domain-specific features utilizing the inherent model capacity via pruning. We also perform effective inference using a novel batch-norm based metric to predict the final model parameters to be used accurately. Our approach achieves not only state-of-the-art performance but also prevents catastrophic forgetting of past domains significantly. Our code is made publicly available.