On the Use of Sparse Filtering for Covariate Shift Adaptation

TL;DR

This paper analyzes the effectiveness of sparse filtering for covariate shift adaptation, introduces a new periodic sparse filtering algorithm, and validates its theoretical and practical performance on synthetic and real data.

Contribution

It provides a theoretical analysis of sparse filtering's limitations and proposes a novel periodic sparse filtering method that relaxes these constraints for better adaptation.

Findings

Sparse filtering performs covariate shift adaptation only under cosine-structured label distributions.

Periodic sparse filtering can adapt under periodic label distribution structures.

Experimental results show the proposed method achieves competitive performance on real datasets.

Abstract

In this paper we formally analyse the use of sparse filtering algorithms to perform covariate shift adaptation. We provide a theoretical analysis of sparse filtering by evaluating the conditions required to perform covariate shift adaptation. We prove that sparse filtering can perform adaptation only if the conditional distribution of the labels has a structure explained by a cosine metric. To overcome this limitation, we propose a new algorithm, named periodic sparse filtering, and carry out the same theoretical analysis regarding covariate shift adaptation. We show that periodic sparse filtering can perform adaptation under the looser and more realistic requirement that the conditional distribution of the labels has a periodic structure, which may be satisfied, for instance, by user-dependent data sets. We experimentally validate our theoretical results on synthetic data. Moreover, we apply periodic sparse filtering to real-world data sets to demonstrate that this simple and computationally efficient algorithm is able to achieve competitive performances.