Marginal and training-conditional guarantees in one-shot federated conformal prediction

TL;DR

This paper develops efficient, distribution-free algorithms for one-shot federated conformal prediction that provide marginal and training-conditional guarantees, achieving performance comparable to centralized methods.

Contribution

Introduces novel, computationally-efficient algorithms for federated conformal prediction with theoretical guarantees and empirical validation, in a one-shot communication setting.

Findings

Algorithms achieve coverage similar to centralized methods.

Theoretical bounds are tight and comparable to centralized case.

Experimental results confirm practical effectiveness.

Abstract

We study conformal prediction in the one-shot federated learning setting. The main goal is to compute marginally and training-conditionally valid prediction sets, at the server-level, in only one round of communication between the agents and the server. Using the quantile-of-quantiles family of estimators and split conformal prediction, we introduce a collection of computationally-efficient and distribution-free algorithms that satisfy the aforementioned requirements. Our approaches come from theoretical results related to order statistics and the analysis of the Beta-Beta distribution. We also prove upper bounds on the coverage of all proposed algorithms when the nonconformity scores are almost surely distinct. For algorithms with training-conditional guarantees, these bounds are of the same order of magnitude as those of the centralized case. Remarkably, this implies that the one-shot federated learning setting entails no significant loss compared to the centralized case. Our experiments confirm that our algorithms return prediction sets with coverage and length similar to those obtained in a centralized setting.