Personalized Federated Learning with Bidirectional Communication Compression via One-Bit Random Sketching

TL;DR

This paper introduces pFed1BS, a personalized federated learning framework that uses one-bit random sketching and the Fast Hadamard Transform to significantly reduce communication costs while maintaining competitive model performance.

Contribution

The paper proposes a novel personalized federated learning method employing extreme communication compression via one-bit sketches and introduces a sign-based regularizer for effective personalization.

Findings

Substantially reduces communication costs in federated learning.

Achieves competitive performance compared to existing communication-efficient algorithms.

Theoretical convergence guarantees to a stationary neighborhood.

Abstract

Federated Learning (FL) enables collaborative training across decentralized data, but faces key challenges of bidirectional communication overhead and client-side data heterogeneity. To address communication costs while embracing data heterogeneity, we propose pFed1BS, a novel personalized federated learning framework that achieves extreme communication compression through one-bit random sketching. In personalized FL, the goal shifts from training a single global model to creating tailored models for each client. In our framework, clients transmit highly compressed one-bit sketches, and the server aggregates and broadcasts a global one-bit consensus. To enable effective personalization, we introduce a sign-based regularizer that guides local models to align with the global consensus while preserving local data characteristics. To mitigate the computational burden of random sketching, we employ the Fast Hadamard Transform for efficient projection. Theoretical analysis guarantees that our algorithm converges to a stationary neighborhood of the global potential function. Numerical simulations demonstrate that pFed1BS substantially reduces communication costs while achieving competitive performance compared to advanced communication-efficient FL algorithms.