Private Federated Learning with Autotuned Compression

TL;DR

This paper introduces adaptive, privacy-preserving compression techniques for federated learning that automatically tune compression rates during training, ensuring efficiency without manual parameter tuning.

Contribution

It presents instance-optimal, on-the-fly compression methods for private federated learning that adapt to problem difficulty while maintaining privacy guarantees.

Findings

Achieves effective compression without tuning on real datasets

Maintains provable privacy guarantees with secure aggregation and differential privacy

Demonstrates instance-optimality in mean estimation

Abstract

We propose new techniques for reducing communication in private federated learning without the need for setting or tuning compression rates. Our on-the-fly methods automatically adjust the compression rate based on the error induced during training, while maintaining provable privacy guarantees through the use of secure aggregation and differential privacy. Our techniques are provably instance-optimal for mean estimation, meaning that they can adapt to the ``hardness of the problem" with minimal interactivity. We demonstrate the effectiveness of our approach on real-world datasets by achieving favorable compression rates without the need for tuning.