Handling Covariate Mismatch in Federated Linear Prediction

TL;DR

This paper addresses the challenge of covariate mismatch in federated linear prediction, proposing modular methods for low and high-dimensional settings, with theoretical guarantees on learning rates.

Contribution

It introduces novel approaches for federated learning with covariate mismatch, including a plug-in estimator and an impute-then-regress strategy, with comprehensive theoretical analysis.

Findings

Effective covariance and cross-moment estimation in low dimensions.

Imputation-based linear modeling in high dimensions.

Explicit learning rate characterizations under covariate mismatch.

Abstract

Federated learning enables institutions to train predictive models collaboratively without sharing raw data, addressing privacy and regulatory constraints. In the standard horizontal setting, clients hold disjoint cohorts of individuals and collaborate to learn a shared predictor. Most existing methods, however, assume that all clients measure the same features. We study the more realistic setting of covariate mismatch, where each client observes a different subset of features, which typically arises in multicenter collaborations with no prior agreement on data collection. We formalize learning a linear prediction under client-wise MCAR patterns and develop two modular approaches tailored to the dimensional regime and communication budget. In the low-dimensional setting, we propose a plug-in estimator that approximates the oracle linear predictor by aggregating sufficient statistics to estimate the covariance and cross-moment terms. In higher dimensions, we study an impute-then-regress strategy: (i) impute missing covariates using any exchangeability-preserving imputation procedure, and (ii) fit a ridge-regularized linear model on the completed data. We provide asymptotic and finite-sample learning rates for our predictors, explicitly characterizing their behaviour with the global dimension, the client-specific feature partition, and the distribution of samples across sites.