Pursuing Overall Welfare in Federated Learning through Sequential Decision Making

TL;DR

This paper introduces AAggFF, an adaptive aggregation method for federated learning that improves client fairness by framing the problem as online convex optimization, with theoretical guarantees and empirical validation.

Contribution

The paper unifies fairness-aware aggregation strategies into an online convex optimization framework and proposes AAggFF with tailored versions for cross-device and cross-silo settings.

Findings

AAggFF achieves better client fairness than existing methods.

Theoretical regret bounds are established for both settings.

Empirical results validate improved fairness performance.

Abstract

In traditional federated learning, a single global model cannot perform equally well for all clients. Therefore, the need to achieve the client-level fairness in federated system has been emphasized, which can be realized by modifying the static aggregation scheme for updating the global model to an adaptive one, in response to the local signals of the participating clients. Our work reveals that existing fairness-aware aggregation strategies can be unified into an online convex optimization framework, in other words, a central server's sequential decision making process. To enhance the decision making capability, we propose simple and intuitive improvements for suboptimal designs within existing methods, presenting AAggFF. Considering practical requirements, we further subdivide our method tailored for the cross-device and the cross-silo settings, respectively. Theoretical analyses guarantee sublinear regret upper bounds for both settings: $\mathcal{O}(\sqrt{T \log{K}})$ for the cross-device setting, and $\mathcal{O}(K \log{T})$ for the cross-silo setting, with $K$ clients and $T$ federation rounds. Extensive experiments demonstrate that the federated system equipped with AAggFF achieves better degree of client-level fairness than existing methods in both practical settings. Code is available at https://github.com/vaseline555/AAggFF