Source Component Shift Adaptation via Offline Decomposition and Online Mixing Approach

TL;DR

This paper introduces a novel approach for source component shift adaptation that combines offline decomposition of source components with online mixing weight updates, leading to significantly improved predictive performance on real-world datasets.

Contribution

It proposes a new method that decomposes source components offline and adapts mixing weights online, effectively handling recurring shifts and outperforming existing methods.

Findings

Reduces cumulative test loss by up to 67.4%

Outperforms baseline methods on various datasets

Leverages theoretical insights for better adaptation

Abstract

This paper addresses source component shift adaptation, aiming to update predictions adapting to source component shifts for incoming data streams based on past training data. Existing online learning methods often fail to utilize recurring shifts effectively, while model-pool-based methods struggle to capture individual source components, leading to poor adaptation. In this paper, we propose a source component shift adaptation method via an offline decomposition and online mixing approach. We theoretically identify that the problem can be divided into two subproblems: offline source component decomposition and online mixing weight adaptation. Based on this, our method first determines prediction models, each of which learns a source component solely based on past training data offline through the EM algorithm. Then, it updates the mixing weight of the prediction models for precise prediction through online convex optimization. Thanks to our theoretical derivation, our method fully leverages the characteristics of the shifts, achieving superior adaptation performance over existing methods. Experiments conducted on various real-world regression datasets demonstrate that our method outperforms baselines, reducing the cumulative test loss by up to 67.4%.