Federated Deep AUC Maximization for Heterogeneous Data with a Constant Communication Complexity

TL;DR

This paper introduces a federated deep AUC maximization algorithm for heterogeneous data that achieves constant communication complexity, significantly improving efficiency over existing methods, and demonstrates its effectiveness on medical imaging datasets.

Contribution

The paper proposes a novel FDAM algorithm with constant communication complexity, applicable to non-convex min-max problems, and demonstrates its superior efficiency and effectiveness in medical imaging applications.

Findings

Communication complexity is constant and independent of the number of machines.

FDAM improves AUC scores in medical chest X-ray classification.

Algorithm outperforms existing methods in heterogeneous federated settings.

Abstract

Deep AUC (area under the ROC curve) Maximization (DAM) has attracted much attention recently due to its great potential for imbalanced data classification. However, the research on Federated Deep AUC Maximization (FDAM) is still limited. Compared with standard federated learning (FL) approaches that focus on decomposable minimization objectives, FDAM is more complicated due to its minimization objective is non-decomposable over individual examples. In this paper, we propose improved FDAM algorithms for heterogeneous data by solving the popular non-convex strongly-concave min-max formulation of DAM in a distributed fashion, which can also be applied to a class of non-convex strongly-concave min-max problems. A striking result of this paper is that the communication complexity of the proposed algorithm is a constant independent of the number of machines and also independent of the accuracy level, which improves an existing result by orders of magnitude. The experiments have demonstrated the effectiveness of our FDAM algorithm on benchmark datasets, and on medical chest X-ray images from different organizations. Our experiment shows that the performance of FDAM using data from multiple hospitals can improve the AUC score on testing data from a single hospital for detecting life-threatening diseases based on chest radiographs. The proposed method is implemented in our open-sourced library LibAUC (www.libauc.org) whose github address is https://github.com/Optimization-AI/ICML2021_FedDeepAUC_CODASCA.