Sampling-based federated inference for M-estimators with non-smooth objective functions

TL;DR

This paper introduces a sampling-based federated inference framework for non-smooth M-estimators, enabling efficient, privacy-preserving statistical inference across multiple data sites with heterogeneity.

Contribution

It develops a novel MCMC-based federated inference method with adaptive site selection and regularization, achieving optimal asymptotic efficiency and robustness against negative transfer.

Findings

Significant improvements in inference accuracy demonstrated in simulations.

Method successfully applied to real-world diabetes data.

Establishes theoretical guarantees of consistency and asymptotic normality.

Abstract

We propose a novel sampling-based federated learning framework for statistical inference on M-estimators with non-smooth objective functions, which frequently arise in modern statistical applications such as quantile regression and AUC maximization. Classical inference methods for such estimators are often computationally intensive or require nonparametric estimation of nuisance quantities. Our approach circumvents these challenges by leveraging Markov Chain Monte Carlo (MCMC) sampling and a second-stage perturbation scheme to efficiently estimate both the parameter of interest and its variance. In the presence of multiple sites with data-sharing constraints, we introduce an adaptive strategy to borrow information from potentially heterogeneous source sites without transferring individual-level data. This strategy selects source sites based on a dissimilarity measure and constructs an optimally weighted estimator using a lasso regularization. The resulting estimator has an oracle property, i.e., it achieves the optimal asymptotical efficiency by borrowing information from eligible sites while guarding against negative transfer. We establish consistency and asymptotic normality of our proposed estimators and validate the method through extensive simulations and a real-data application on type 2 diabetes. Our results demonstrate substantial gains in inference precision and underscore the importance of inclusive, data-adaptive analysis frameworks in federated learning settings.