RPN: A Residual Pooling Network for Efficient Federated Learning

TL;DR

This paper introduces Residual Pooling Network (RPN), a novel compression method that enhances communication efficiency in federated learning while maintaining comparable model performance, enabling easier real-world deployment.

Contribution

The paper proposes RPN, an end-to-end compression strategy for federated learning that reduces data transmission without sacrificing accuracy, simplifying deployment.

Findings

RPN significantly reduces communication costs in federated learning.

RPN achieves comparable performance to standard federated learning.

RPN is applicable to CNN-based models in various scenarios.

Abstract

Federated learning is a distributed machine learning framework which enables different parties to collaboratively train a model while protecting data privacy and security. Due to model complexity, network unreliability and connection in-stability, communication cost has became a major bottleneck for applying federated learning to real-world applications. Current existing strategies are either need to manual setting for hyperparameters, or break up the original process into multiple steps, which make it hard to realize end-to-end implementation. In this paper, we propose a novel compression strategy called Residual Pooling Network (RPN). Our experiments show that RPN not only reduce data transmission effectively, but also achieve almost the same performance as compared to standard federated learning. Our new approach performs as an end-to-end procedure, which should be readily applied to all CNN-based model training scenarios for improvement of communication efficiency, and hence make it easy to deploy in real-world application without much human intervention.