BIPPO: Budget-Aware Independent PPO for Energy-Efficient Federated Learning Services

TL;DR

BIPPO is a novel energy-efficient multi-agent RL method designed for resource-constrained federated learning, significantly improving accuracy and scalability while maintaining low energy consumption in IoT environments.

Contribution

This paper introduces BIPPO, a budget-aware RL approach tailored for energy-efficient federated learning, addressing infrastructure challenges and enhancing client selection performance.

Findings

BIPPO increases mean accuracy over non-RL and traditional methods.

BIPPO maintains low energy consumption regardless of client number.

BIPPO performs well on non-IID data in budget-constrained settings.

Abstract

Federated Learning (FL) is a promising machine learning solution in large-scale IoT systems, guaranteeing load distribution and privacy. However, FL does not natively consider infrastructure efficiency, a critical concern for systems operating in resource-constrained environments. Several Reinforcement Learning (RL) based solutions offer improved client selection for FL; however, they do not consider infrastructure challenges, such as resource limitations and device churn. Furthermore, the training of RL methods is often not designed for practical application, as these approaches frequently do not consider generalizability and are not optimized for energy efficiency. To fill this gap, we propose BIPPO (Budget-aware Independent Proximal Policy Optimization), which is an energy-efficient multi-agent RL solution that improves performance. We evaluate BIPPO on two image classification tasks run in a highly budget-constrained setting, with FL clients training on non-IID data, a challenging context for vanilla FL. The improved sampler of BIPPO enables it to increase the mean accuracy compared to non-RL mechanisms, traditional PPO, and IPPO. In addition, BIPPO only consumes a negligible proportion of the budget, which stays consistent even if the number of clients increases. Overall, BIPPO delivers a performant, stable, scalable, and sustainable solution for client selection in IoT-FL.