Towards regularized learning from functional data with covariate shift

TL;DR

This paper develops a regularization framework for unsupervised domain adaptation in vector-valued regression under covariate shift, with theoretical guarantees and practical algorithms, demonstrated on face image data.

Contribution

It introduces a novel operator learning algorithm in vRKHS for covariate shift, with aggregation methods for tuning parameter selection and theoretical convergence analysis.

Findings

The method achieves optimal convergence rates.

Aggregation improves estimator robustness.

Effective on real-world face image dataset.

Abstract

This paper investigates a general regularization framework for unsupervised domain adaptation in vector-valued regression under the covariate shift assumption, utilizing vector-valued reproducing kernel Hilbert spaces (vRKHS). Covariate shift occurs when the input distributions of the training and test data differ, introducing significant challenges for reliable learning. By restricting the hypothesis space, we develop a practical operator learning algorithm capable of handling functional outputs. We establish optimal convergence rates for the proposed framework under a general source condition, providing a theoretical foundation for regularized learning in this setting. We also propose an aggregation-based approach that forms a linear combination of estimators corresponding to different regularization parameters and different kernels. The proposed approach addresses the challenge of selecting appropriate tuning parameters, which is crucial for constructing a good estimator, and we provide a theoretical justification for its effectiveness. Furthermore, we illustrate the proposed method on a real-world face image dataset, demonstrating robustness and effectiveness in mitigating distributional discrepancies under covariate shift.