Graph Federated Learning with Hidden Representation Sharing

TL;DR

This paper introduces Graph Federated Learning (GFL), combining Learning on Graphs and Federated Learning by sharing hidden representations to enhance privacy and performance in multi-client graph tasks.

Contribution

The paper formulates GFL to unify LoG and FL, proposes sharing hidden representations for privacy, and develops a gradient estimation method with convergence analysis.

Findings

Effective GFL method for graph classification tasks.

Theoretical convergence guarantees under non-convex objectives.

Experimental results align with theoretical predictions.

Abstract

Learning on Graphs (LoG) is widely used in multi-client systems when each client has insufficient local data, and multiple clients have to share their raw data to learn a model of good quality. One scenario is to recommend items to clients with limited historical data and sharing similar preferences with other clients in a social network. On the other hand, due to the increasing demands for the protection of clients' data privacy, Federated Learning (FL) has been widely adopted: FL requires models to be trained in a multi-client system and restricts sharing of raw data among clients. The underlying potential data-sharing conflict between LoG and FL is under-explored and how to benefit from both sides is a promising problem. In this work, we first formulate the Graph Federated Learning (GFL) problem that unifies LoG and FL in multi-client systems and then propose sharing hidden representation instead of the raw data of neighbors to protect data privacy as a solution. To overcome the biased gradient problem in GFL, we provide a gradient estimation method and its convergence analysis under the non-convex objective. In experiments, we evaluate our method in classification tasks on graphs. Our experiment shows a good match between our theory and the practice.