FL Games: A federated learning framework for distribution shifts

TL;DR

FL Games introduces a game-theoretic federated learning framework that focuses on learning invariant causal features to improve generalization across diverse, non-i.i.d. client data distributions, reducing communication and handling heterogeneity.

Contribution

The paper proposes FL Games, a novel game-theoretic approach for federated learning that learns invariant causal features, addressing distribution shifts and improving out-of-distribution generalization.

Findings

Achieves high out-of-distribution performance on benchmarks.

Scales well with number of clients and fewer communication rounds.

Effectively learns stable, invariant features across clients.

Abstract

Federated learning aims to train predictive models for data that is distributed across clients, under the orchestration of a server. However, participating clients typically each hold data from a different distribution, whereby predictive models with strong in-distribution generalization can fail catastrophically on unseen domains. In this work, we argue that in order to generalize better across non-i.i.d. clients, it is imperative to only learn correlations that are stable and invariant across domains. We propose FL Games, a game-theoretic framework for federated learning for learning causal features that are invariant across clients. While training to achieve the Nash equilibrium, the traditional best response strategy suffers from high-frequency oscillations. We demonstrate that FL Games effectively resolves this challenge and exhibits smooth performance curves. Further, FL Games scales well in the number of clients, requires significantly fewer communication rounds, and is agnostic to device heterogeneity. Through empirical evaluation, we demonstrate that FL Games achieves high out-of-distribution performance on various benchmarks.