ProFL: Performative Robust Optimal Federated Learning

TL;DR

This paper introduces a new federated learning algorithm that effectively finds performative optimal points despite data noise and distribution shifts, improving robustness and convergence in realistic scenarios.

Contribution

It proposes Performative Robust Optimal Federated Learning, overcoming previous limitations by handling noisy data and non-convex objectives, with proven convergence analysis.

Findings

Outperforms state-of-the-art methods in experiments

Handles noisy and contaminated data effectively

Converges under Polyak-Lojasiewicz condition

Abstract

Performative prediction is a framework that captures distribution shifts that occur during the training of machine learning models due to their deployment. As the trained model is used, data generation causes the model to evolve, leading to deviations from the original data distribution. The impact of such model-induced distribution shifts in federated learning is increasingly likely to transpire in real-life use cases. A recently proposed approach extends performative prediction to federated learning with the resulting model converging to a performative stable point, which may be far from the performative optimal point. Earlier research in centralized settings has shown that the performative optimal point can be achieved under model-induced distribution shifts, but these approaches require the performative risk to be convex and the training data to be noiseless, assumptions often violated in realistic federated learning systems. This paper overcomes all of these shortcomings and proposes Performative Robust Optimal Federated Learning, an algorithm that finds performative optimal points in federated learning from noisy and contaminated data. We present the convergence analysis under the Polyak-Lojasiewicz condition, which applies to non-convex objectives. Extensive experiments on multiple datasets demonstrate the advantage of Robust Optimal Federated Learning over the state-of-the-art.