DPAUC: Differentially Private AUC Computation in Federated Learning

TL;DR

This paper introduces a new algorithm for accurately computing the AUC metric in federated learning while preserving label privacy through differential privacy, addressing privacy concerns during model evaluation.

Contribution

It presents the first evaluation algorithm for AUC in federated learning that maintains label differential privacy, ensuring privacy during model assessment.

Findings

Accurately computes AUC under label differential privacy.

Maintains high accuracy compared to ground truth.

Demonstrates effectiveness through extensive experiments.

Abstract

Federated learning (FL) has gained significant attention recently as a privacy-enhancing tool to jointly train a machine learning model by multiple participants. The prior work on FL has mostly studied how to protect label privacy during model training. However, model evaluation in FL might also lead to potential leakage of private label information. In this work, we propose an evaluation algorithm that can accurately compute the widely used AUC (area under the curve) metric when using the label differential privacy (DP) in FL. Through extensive experiments, we show our algorithms can compute accurate AUCs compared to the ground truth. The code is available at {\url{https://github.com/bytedance/fedlearner/tree/master/example/privacy/DPAUC}}.