Multi-objective Deep Reinforcement Learning for Mobile Edge Computing

TL;DR

This paper presents a multi-objective deep reinforcement learning approach for mobile edge computing that optimizes energy and delay without prior knowledge of preferences, significantly improving Pareto front quality.

Contribution

The study introduces a novel MORL-based resource scheduling scheme with a new state encoding and reward function for MEC systems, addressing unknown application preferences.

Findings

Enhanced Pareto front hypervolume by up to 233.1%

Effective handling of unknown preferences in MEC scheduling

Improved long-term energy and delay performance

Abstract

Mobile edge computing (MEC) is essential for next-generation mobile network applications that prioritize various performance metrics, including delays and energy consumption. However, conventional single-objective scheduling solutions cannot be directly applied to practical systems in which the preferences of these applications (i.e., the weights of different objectives) are often unknown or challenging to specify in advance. In this study, we address this issue by formulating a multi-objective offloading problem for MEC with multiple edges to minimize expected long-term energy consumption and transmission delay while considering unknown preferences as parameters. To address the challenge of unknown preferences, we design a multi-objective (deep) reinforcement learning (MORL)-based resource scheduling scheme with proximal policy optimization (PPO). In addition, we introduce a well-designed state encoding method for constructing features for multiple edges in MEC systems, a sophisticated reward function for accurately computing the utilities of delay and energy consumption. Simulation results demonstrate that our proposed MORL scheme enhances the hypervolume of the Pareto front by up to 233.1% compared to benchmarks. Our full framework is available at https://github.com/gracefulning/mec_morl_multipolicy.