HERL: Tiered Federated Learning with Adaptive Homomorphic Encryption using Reinforcement Learning

TL;DR

HERL introduces a reinforcement learning-based method to adaptively optimize homomorphic encryption parameters in federated learning, reducing computational overhead while maintaining model utility and security across heterogeneous client environments.

Contribution

HERL is the first to apply reinforcement learning for dynamic encryption parameter tuning in tiered federated learning, improving efficiency and security adaptation.

Findings

Utility improved by 17%

Convergence time reduced by 24%

Convergence efficiency increased by 30%

Abstract

Federated Learning is a well-researched approach for collaboratively training machine learning models across decentralized data while preserving privacy. However, integrating Homomorphic Encryption to ensure data confidentiality introduces significant computational and communication overheads, particularly in heterogeneous environments where clients have varying computational capacities and security needs. In this paper, we propose HERL, a Reinforcement Learning-based approach that uses Q-Learning to dynamically optimize encryption parameters, specifically the polynomial modulus degree, $N$, and the coefficient modulus, $q$, across different client tiers. Our proposed method involves first profiling and tiering clients according to the chosen clustering approach, followed by dynamically selecting the most suitable encryption parameters using an RL-agent. Experimental results demonstrate that our approach significantly reduces the computational overhead while maintaining utility and a high level of security. Empirical results show that HERL improves utility by 17%, reduces the convergence time by up to 24%, and increases convergence efficiency by up to 30%, with minimal security loss.