Federated Learning Meets Multi-objective Optimization

TL;DR

This paper introduces FedMGDA+, a novel federated learning algorithm formulated as a multi-objective optimization problem, ensuring fairness and robustness while guaranteeing convergence to Pareto stationary solutions.

Contribution

It proposes FedMGDA+, a simple, hyperparameter-efficient algorithm for federated learning that balances user fairness and robustness, with proven convergence and superior performance.

Findings

FedMGDA+ converges to Pareto stationary solutions.

The algorithm outperforms state-of-the-art methods.

FedMGDA+ maintains performance without sacrificing user fairness.

Abstract

Federated learning has emerged as a promising, massively distributed way to train a joint deep model over large amounts of edge devices while keeping private user data strictly on device. In this work, motivated from ensuring fairness among users and robustness against malicious adversaries, we formulate federated learning as multi-objective optimization and propose a new algorithm FedMGDA+ that is guaranteed to converge to Pareto stationary solutions. FedMGDA+ is simple to implement, has fewer hyperparameters to tune, and refrains from sacrificing the performance of any participating user. We establish the convergence properties of FedMGDA+ and point out its connections to existing approaches. Extensive experiments on a variety of datasets confirm that FedMGDA+ compares favorably against state-of-the-art.