Trustworthy Federated Label Distribution Learning under Annotation Quality Disparity

TL;DR

This paper introduces FedQual, a framework for federated label distribution learning that accounts for annotation quality disparities across clients, improving robustness and reliability in privacy-sensitive applications.

Contribution

FedQual is a novel quality-aware federated LDL framework with adaptive client training and reliability-aware aggregation, addressing annotation quality heterogeneity.

Findings

FedQual outperforms baseline methods on new benchmarks with annotation quality disparity.

Client-specific calibration improves learning under heterogeneous supervision quality.

Theoretical analysis confirms calibration's advantage over uniform approaches.

Abstract

Label Distribution Learning (LDL) models supervision as an instance-wise probability distribution, enabling fine-grained learning under inherent ambiguity, but its success relies on high-fidelity label distributions that are costly to obtain and thus often noisy. Motivated by privacy-sensitive applications, we study Federated Label Distribution Learning (Fed-LDL), where data isolation further induces heterogeneous annotation quality across clients, making local updates unevenly reliable and breaking sample-size-based aggregation (e.g., FedAvg). To address this trust dilemma, we propose FedQual, a quality-aware Fed-LDL framework with two coupled mechanisms: (i) quality-adaptive client training guided by a global semantic anchor that calibrates low-quality clients while preserving high-quality autonomy, and (ii) reliability-aware server aggregation that reweights client contributions by effective reliable information rather than raw sample size. To enable rigorous evaluation, we construct four new Fed-LDL benchmarks (FER-LDL, FI-LDL, PIPAL-LDL, and KADID-LDL) with controlled annotation quality disparity. We further provide a theoretical guarantee showing that under heterogeneous supervision quality, client-specific calibration is strictly better than any uniform calibration. Extensive experiments on the proposed benchmarks demonstrate the effectiveness of FedQual.