No Free Lunch: Balancing Learning and Exploitation at the Network Edge

TL;DR

This paper examines the trade-offs of deploying deep reinforcement learning for network optimization at the edge, highlighting the significant resource costs of learning that can impact performance evaluations.

Contribution

It introduces an optimization framework that accounts for learning costs in resource-constrained network edge environments, emphasizing the importance of considering these costs.

Findings

Learning costs can be critical in resource-limited edge networks.

Ignoring learning costs can lead to overestimating DRL performance.

Resource trade-offs influence the effectiveness of DRL-based strategies.

Abstract

Over the last few years, the DRL paradigm has been widely adopted for 5G and beyond network optimization because of its extreme adaptability to many different scenarios. However, collecting and processing learning data entail a significant cost in terms of communication and computational resources, which is often disregarded in the networking literature. In this work, we analyze the cost of learning in a resource-constrained system, defining an optimization problem in which training a DRL agent makes it possible to improve the resource allocation strategy but also reduces the number of available resources. Our simulation results show that the cost of learning can be critical when evaluating DRL schemes on the network edge and that assuming a cost-free learning model can lead to significantly overestimating performance.