Conformalized Neural Networks for Federated Uncertainty Quantification under Dual Heterogeneity

TL;DR

This paper introduces FedWQ-CP, a federated learning approach that uses conformal prediction to provide reliable uncertainty quantification across heterogeneous agents with minimal communication, ensuring coverage and efficiency.

Contribution

The paper proposes FedWQ-CP, a novel federated conformal prediction method that achieves joint coverage under dual heterogeneity with single-round calibration and minimal communication.

Findings

Maintains agent-wise and global coverage in heterogeneous settings.

Produces the smallest prediction sets or intervals among compared methods.

Validated on seven public datasets for classification and regression.

Abstract

Federated learning (FL) faces challenges in uncertainty quantification (UQ). Without reliable UQ, FL systems risk deploying overconfident models at under-resourced agents, leading to silent local failures despite seemingly satisfactory global performance. Existing federated UQ approaches often address data heterogeneity or model heterogeneity in isolation, overlooking their joint effect on coverage reliability across agents. Conformal prediction is a widely used distribution-free UQ framework, yet its applications in heterogeneous FL settings remains underexplored. We provide FedWQ-CP, a simple yet effective approach that balances empirical coverage performance with efficiency at both global and agent levels under the dual heterogeneity. FedWQ-CP performs agent-server calibration in a single communication round. On each agent, conformity scores are computed on calibration data and a local quantile threshold is derived. Each agent then transmits only its quantile threshold and calibration sample size to the server. The server simply aggregates these thresholds through a weighted average to produce a global threshold. Experimental results on seven public datasets for both classification and regression demonstrate that FedWQ-CP empirically maintains agent-wise and global coverage while producing the smallest prediction sets or intervals.