# Multi-Armed Bandit for Energy-Efficient and Delay-Sensitive Edge Computing in Dynamic Networks with Uncertainty

## TL;DR

This paper introduces a multi-armed bandit approach for energy-efficient, delay-sensitive server selection in dynamic edge computing environments, addressing uncertainty and abrupt changes to optimize latency and energy use.

## Contribution

It formulates the server selection as a budget-limited multi-armed bandit problem with time-varying statistics and proposes a novel policy with proven regret bounds.

## Key findings

- Proposed method outperforms existing solutions in simulations.
- Effective handling of environment dynamics and abrupt changes.
- Improved energy and delay performance in edge computing scenarios.

## Abstract

In the edge computing paradigm, mobile devices offload the computational tasks to an edge server by routing the required data over the wireless network. The full potential of edge computing becomes realized only if a smart device selects the most appropriate server in terms of the latency and energy consumption, among many available ones. The server selection problem is challenging due to the randomness of the environment and lack of prior information about the environment. Therefore, a smart device, which sequentially chooses a server under uncertainty, aims to improve its decision based on the historical time and energy consumption. The problem becomes more complicated in a dynamic environment, where key variables might undergo abrupt changes. To deal with the aforementioned problem, we first analyze the required time and energy to data transmission and processing. We then use the analysis to cast the problem as a budget-limited multi-armed bandit problem, where each arm is associated with a reward and cost, with time-variant statistical characteristics. We propose a policy to solve the formulated problem and prove a regret bound. The numerical results demonstrate the superiority of the proposed method compared to a number of existing solutions.