Pseudo-Labeling for Unsupervised Domain Adaptation with Kernel GLMs

TL;DR

This paper introduces a kernel GLM-based framework for unsupervised domain adaptation under covariate shift, using pseudo-labeling and model selection to improve prediction accuracy in the target domain.

Contribution

It presents a novel principled approach combining pseudo-labeling and model selection within kernel GLMs for covariate shift adaptation, with theoretical guarantees.

Findings

Consistent performance improvements over source-only methods.

Theoretical non-asymptotic excess-risk bounds established.

Effective labeled sample size quantifies adaptation performance.

Abstract

We propose a principled framework for unsupervised domain adaptation under covariate shift in kernel Generalized Linear Models (GLMs), encompassing kernelized linear, logistic, and Poisson regression with ridge regularization. Our goal is to minimize prediction error in the target domain by leveraging labeled source data and unlabeled target data, despite differences in covariate distributions. We partition the labeled source data into two batches: one for training a family of candidate models, and the other for building an imputation model. This imputation model generates pseudo-labels for the target data, enabling robust model selection. We establish non-asymptotic excess-risk bounds that characterize adaptation performance through an "effective labeled sample size", explicitly accounting for the unknown covariate shift. Experiments on synthetic and real datasets demonstrate consistent performance gains over source-only baselines.