Learning-Based Computation Offloading for IoT Devices with Energy Harvesting

TL;DR

This paper introduces reinforcement learning methods, including hotbooting Q-learning and deep Q-networks, to optimize computation offloading for energy-harvesting IoT devices in wireless networks with multiple MEC servers, improving efficiency and reducing delays.

Contribution

It proposes novel RL-based offloading schemes that do not require prior MEC model knowledge, accelerating learning with hotbooting and deep learning techniques.

Findings

Fast DQN scheme reduces energy consumption and delay.

Proposed methods achieve near-optimal offloading policies.

Performance bounds are established for typical MEC scenarios.

Abstract

Internet of Things (IoT) devices can apply mobile-edge computing (MEC) and energy harvesting (EH) to provide the satisfactory quality of experiences for computation intensive applications and prolong the battery lifetime. In this article, we investigate the computation offloading for IoT devices with energy harvesting in wireless networks with multiple MEC devices such as base stations and access points, each with different computation resource and radio communication capability. We propose a reinforcement learning based computation offloading framework for an IoT device to choose the MEC device and determine the offloading rate according to the current battery level, the previous radio bandwidth to each MEC device and the predicted amount of the harvested energy. A "hotbooting" Q-learning based computation offloading scheme is proposed for an IoT device to achieve the optimal offloading performance without being aware of the MEC model, the energy consumption and computation latency model. We also propose a fast deep Q-network (DQN) based offloading scheme, which combines the deep learning and hotbooting techniques to accelerate the learning speed of Q-learning. We show that the proposed schemes can achieve the optimal offloading policy after sufficiently long learning time and provide their performance bounds under two typical MEC scenarios. Simulations are performed for IoT devices that use wireless power transfer to capture the ambient radio-frequency signals to charge the IoT batteries. Simulation results show that the fast DQN-based offloading scheme reduces the energy consumption, decreases the computation delay and the task drop ratio, and increases the utility of the IoT device in dynamic MEC, compared with the benchmark Q-learning based offloading.