Gradients as an Action: Towards Communication-Efficient Federated Recommender Systems via Adaptive Action Sharing

TL;DR

This paper introduces FedRAS, a federated recommender system framework that reduces communication costs by sharing gradient actions instead of full item embeddings, maintaining performance across heterogeneous environments.

Contribution

FedRAS proposes an adaptive action-sharing strategy that clusters gradients into actions for efficient communication, addressing high overhead and low efficiency in federated recommender systems.

Findings

Reduces communication payloads by up to 96.88%.

Maintains recommendation performance across heterogeneous scenarios.

Uses adaptive clustering to dynamically adjust the number of actions.

Abstract

As a promising privacy-aware collaborative model training paradigm, Federated Learning (FL) is becoming popular in the design of distributed recommender systems. However, Federated Recommender Systems (FedRecs) greatly suffer from two major problems: i) extremely high communication overhead due to massive item embeddings involved in recommendation systems, and ii) intolerably low training efficiency caused by the entanglement of both heterogeneous network environments and client devices. Although existing methods attempt to employ various compression techniques to reduce communication overhead, due to the parameter errors introduced by model compression, they inevitably suffer from model performance degradation. To simultaneously address the above problems, this paper presents a communication-efficient FedRec framework named FedRAS, which adopts an action-sharing strategy to cluster the gradients of item embedding into a specific number of model updating actions for communication rather than directly compressing the item embeddings. In this way, the cloud server can use the limited actions from clients to update all the items. Since gradient values are significantly smaller than item embeddings, constraining the directions of gradients (i.e., the action space) introduces smaller errors compared to compressing the entire item embedding matrix into a reduced space. To accommodate heterogeneous devices and network environments, FedRAS incorporates an adaptive clustering mechanism that dynamically adjusts the number of actions. Comprehensive experiments on well-known datasets demonstrate that FedRAS can reduce the size of communication payloads by up to 96.88%, while not sacrificing recommendation performance within various heterogeneous scenarios. We have open-sourced FedRAS at https://github.com/mastlab-T3S/FedRAS.