Agile Reinforcement Learning for Real-Time Task Scheduling in Edge Computing

TL;DR

This paper introduces Agile Reinforcement Learning (aRL), a novel scheduling approach for edge computing that improves adaptation speed and scheduling efficiency for soft real-time applications by using informed exploration and action-masking.

Contribution

The paper proposes aRL, a reinforcement learning method that enhances scheduling speed and accuracy in dynamic edge computing environments through informed exploration.

Findings

aRL achieves higher hit-ratio than baseline methods

aRL converges faster in task scheduling scenarios

Informed exploration improves RL adaptability

Abstract

Soft real-time applications are becoming increasingly complex, posing significant challenges for scheduling offloaded tasks in edge computing environments while meeting task timing constraints. Moreover, the exponential growth of the search space, presence of multiple objectives and parameters, and highly dynamic nature of edge computing environments further exacerbate the complexity of task scheduling. As a result, schedulers based on heuristic and metaheuristic algorithms frequently encounter difficulties in generating optimal or near-optimal task schedules due to their constrained ability to adapt to the dynamic conditions and complex environmental characteristics of edge computing. Accordingly, reinforcement learning algorithms have been incorporated into schedulers to address the complexity and dynamic conditions inherent in task scheduling in edge computing. However, a significant limitation of reinforcement learning algorithms is the prolonged learning time required to adapt to new environments and to address medium- and large-scale problems. This challenge arises from the extensive global action space and frequent random exploration of irrelevant actions. Therefore, this study proposes Agile Reinforcement learning (aRL), in which the RL-agent performs informed exploration and executes only relevant actions. Consequently, the predictability of the RL-agent is enhanced, leading to rapid adaptation and convergence, which positions aRL as a suitable candidate for scheduling the tasks of soft real-time applications in edge computing. The experiments demonstrate that the combination of informed exploration and action-masking methods enables aRL to achieve a higher hit-ratio and converge faster than the baseline approaches.