# Communication-efficient Federated Learning with Single-Step Synthetic Features Compressor for Faster Convergence

## TL;DR

This paper introduces 3SFC, a novel communication-efficient federated learning method that constructs tiny synthetic datasets for faster convergence with minimal data samples, improving robustness and reducing communication costs.

## Contribution

The paper proposes 3SFC, a new single-step synthetic features compressor that achieves ultra-low compression rates and faster convergence in federated learning.

## Key findings

- 3SFC outperforms competing methods in convergence speed.
- It achieves compression rates as low as 0.02%.
- Experimental results validate its effectiveness across multiple datasets.

## Abstract

Reducing communication overhead in federated learning (FL) is challenging but crucial for large-scale distributed privacy-preserving machine learning. While methods utilizing sparsification or others can largely lower the communication overhead, the convergence rate is also greatly compromised. In this paper, we propose a novel method, named single-step synthetic features compressor (3SFC), to achieve communication-efficient FL by directly constructing a tiny synthetic dataset based on raw gradients. Thus, 3SFC can achieve an extremely low compression rate when the constructed dataset contains only one data sample. Moreover, 3SFC's compressing phase utilizes a similarity-based objective function so that it can be optimized with just one step, thereby considerably improving its performance and robustness. In addition, to minimize the compressing error, error feedback (EF) is also incorporated into 3SFC. Experiments on multiple datasets and models suggest that 3SFC owns significantly better convergence rates compared to competing methods with lower compression rates (up to 0.02%). Furthermore, ablation studies and visualizations show that 3SFC can carry more information than competing methods for every communication round, further validating its effectiveness.