Learning to Collaborate Over Graphs: A Selective Federated Multi-Task Learning Approach

TL;DR

This paper introduces a communication-efficient federated multi-task learning method that models client collaboration as a dynamic graph, using community detection to enhance personalization and performance across heterogeneous datasets.

Contribution

The paper proposes a novel graph-based federated learning approach with feature anchors and community detection to improve personalization and collaboration efficiency.

Findings

Outperforms state-of-the-art baselines on heterogeneous datasets.

Demonstrates superior computation and communication efficiency.

Promotes fairness across clients.

Abstract

We present a novel federated multi-task learning method that leverages cross-client similarity to enable personalized learning for each client. To avoid transmitting the entire model to the parameter server, we propose a communication-efficient scheme that introduces a feature anchor, a compact vector representation that summarizes the features learned from the client's local classes. This feature anchor is shared with the server to account for local clients' distribution. In addition, the clients share the classification heads, a lightweight linear layer, and perform a graph-based regularization to enable collaboration among clients. By modeling collaboration between clients as a dynamic graph and continuously updating and refining this graph, we can account for any drift from the clients. To ensure beneficial knowledge transfer and prevent negative collaboration, we leverage a community detection-based approach that partitions this dynamic graph into homogeneous communities, maximizing the sum of task similarities, represented as the graph edges' weights, within each community. This mechanism restricts collaboration to highly similar clients within their formed communities, ensuring positive interaction and preserving personalization. Extensive experiments on two heterogeneous datasets demonstrate that our method significantly outperforms state-of-the-art baselines. Furthermore, we show that our method exhibits superior computation and communication efficiency and promotes fairness across clients.