Collaboration Equilibrium in Federated Learning

TL;DR

This paper introduces the concept of collaboration equilibrium in federated learning, where clients form optimal smaller coalitions based on benefit analysis to improve model performance while preserving privacy.

Contribution

It proposes the benefit graph and Pareto optimization approach to identify stable collaboration coalitions in federated learning, with theoretical guarantees and empirical validation.

Findings

Effective identification of collaboration coalitions.

Theoretical proof of reaching collaboration equilibrium.

Successful experiments on synthetic and real data.

Abstract

Federated learning (FL) refers to the paradigm of learning models over a collaborative research network involving multiple clients without sacrificing privacy. Recently, there have been rising concerns on the distributional discrepancies across different clients, which could even cause counterproductive consequences when collaborating with others. While it is not necessarily that collaborating with all clients will achieve the best performance, in this paper, we study a rational collaboration called ``collaboration equilibrium'' (CE), where smaller collaboration coalitions are formed. Each client collaborates with certain members who maximally improve the model learning and isolates the others who make little contribution. We propose the concept of benefit graph which describes how each client can benefit from collaborating with other clients and advance a Pareto optimization approach to identify the optimal collaborators. Then we theoretically prove that we can reach a CE from the benefit graph through an iterative graph operation. Our framework provides a new way of setting up collaborations in a research network. Experiments on both synthetic and real world data sets are provided to demonstrate the effectiveness of our method.