Overcoming Concept Shift in Domain-Aware Settings through Consolidated Internal Distributions

TL;DR

This paper introduces a method to adapt pre-trained models to concept shifts using internal distribution consolidation, avoiding full retraining and working with unannotated data, supported by theoretical analysis and experiments.

Contribution

It presents a novel domain adaptation algorithm that consolidates internal distributions to handle concept shift without retraining from scratch.

Findings

Effective in improving model performance under concept shift

Works with unannotated initial concept samples

Validated through extensive experiments

Abstract

We develop an algorithm to improve the performance of a pre-trained model under concept shift without retraining the model from scratch when only unannotated samples of initial concepts are accessible. We model this problem as a domain adaptation problem, where the source domain data is inaccessible during model adaptation. The core idea is based on consolidating the intermediate internal distribution, learned to represent the source domain data, after adapting the model. We provide theoretical analysis and conduct extensive experiments to demonstrate that the proposed method is effective.