Conformal Prediction for Federated Uncertainty Quantification Under Label Shift

TL;DR

This paper introduces a federated conformal prediction method that ensures valid uncertainty quantification under label shift and privacy constraints, advancing federated learning's reliability.

Contribution

It develops a novel federated conformal prediction approach using quantile regression with importance weighting, addressing label shift and privacy guarantees.

Findings

Outperforms existing methods in experimental evaluations.

Provides theoretical guarantees for coverage and privacy.

Effectively handles label shift in federated settings.

Abstract

Federated Learning (FL) is a machine learning framework where many clients collaboratively train models while keeping the training data decentralized. Despite recent advances in FL, the uncertainty quantification topic (UQ) remains partially addressed. Among UQ methods, conformal prediction (CP) approaches provides distribution-free guarantees under minimal assumptions. We develop a new federated conformal prediction method based on quantile regression and take into account privacy constraints. This method takes advantage of importance weighting to effectively address the label shift between agents and provides theoretical guarantees for both valid coverage of the prediction sets and differential privacy. Extensive experimental studies demonstrate that this method outperforms current competitors.