A Stochastic Optimization Framework for Private and Fair Learning From Decentralized Data

TL;DR

This paper introduces a novel federated learning algorithm that ensures strong privacy and fairness guarantees, achieving better fairness-accuracy tradeoffs across various privacy levels in decentralized data settings.

Contribution

The paper develops a new private and fair federated learning algorithm satisfying inter-silo record-level differential privacy with convergence guarantees under mild assumptions.

Findings

Achieves state-of-the-art fairness-accuracy tradeoffs.

Guarantees convergence under mild smoothness assumptions.

First convergence guarantee for ISRL-DP nonconvex-strongly concave min-max FL.

Abstract

Machine learning models are often trained on sensitive data (e.g., medical records and race/gender) that is distributed across different "silos" (e.g., hospitals). These federated learning models may then be used to make consequential decisions, such as allocating healthcare resources. Two key challenges emerge in this setting: (i) maintaining the privacy of each person's data, even if other silos or an adversary with access to the central server tries to infer this data; (ii) ensuring that decisions are fair to different demographic groups (e.g., race/gender). In this paper, we develop a novel algorithm for private and fair federated learning (FL). Our algorithm satisfies inter-silo record-level differential privacy (ISRL-DP), a strong notion of private FL requiring that silo i's sent messages satisfy record-level differential privacy for all i. Our framework can be used to promote different fairness notions, including demographic parity and equalized odds. We prove that our algorithm converges under mild smoothness assumptions on the loss function, whereas prior work required strong convexity for convergence. As a byproduct of our analysis, we obtain the first convergence guarantee for ISRL-DP nonconvex-strongly concave min-max FL. Experiments demonstrate the state-of-the-art fairness-accuracy tradeoffs of our algorithm across different privacy levels.