TrustFed: Enabling Trustworthy Medical AI under Data Privacy Constraints

TL;DR

TrustFed is a federated learning framework that provides reliable uncertainty quantification for medical AI across diverse, privacy-sensitive healthcare datasets, addressing heterogeneity and class imbalance issues.

Contribution

It introduces a novel, distribution-free uncertainty quantification method with representation-aware client assignment and soft-threshold aggregation for federated medical imaging.

Findings

Achieved robust coverage guarantees across six imaging modalities.

Demonstrated effectiveness on over 430,000 medical images.

Validated clinically meaningful uncertainty calibration in heterogeneous datasets.

Abstract

Protecting patient privacy remains a fundamental barrier to scaling machine learning across healthcare institutions, where centralizing sensitive data is often infeasible due to ethical, legal, and regulatory constraints. Federated learning offers a promising alternative by enabling privacy-preserving, multi-institutional training without sharing raw patient data; however, real-world deployments face severe challenges from data heterogeneity, site-specific biases, and class imbalance, which degrade predictive reliability and render existing uncertainty quantification methods ineffective. Here, we present TrustFed, a federated uncertainty quantification framework that provides distribution-free, finite-sample coverage guarantees under heterogeneous and imbalanced healthcare data, without requiring centralized access. TrustFed introduces a representation-aware client assignment mechanism that leverages internal model representations to enable effective calibration across institutions, along with a soft-nearest threshold aggregation strategy that mitigates assignment uncertainty while producing compact and reliable prediction sets. Using over 430,000 medical images across six clinically distinct imaging modalities, we conduct one of the most comprehensive evaluations of uncertainty-aware federated learning in medical imaging, demonstrating robust coverage guarantees across datasets with diverse class cardinalities and imbalance regimes. By validating TrustFed at this scale and breadth, our study advances uncertainty-aware federated learning from proof-of-concept toward clinically meaningful, modality-agnostic deployment, positioning statistically guaranteed uncertainty as a core requirement for next-generation healthcare AI systems.