Hybrid Learning for Cold-Start-Aware Microservice Scheduling in Dynamic Edge Environments

TL;DR

This paper introduces a hybrid learning framework combining offline imitation learning and online reinforcement learning to improve microservice scheduling at edge nodes, addressing cold-start issues and resource dynamics.

Contribution

It proposes a novel hybrid learning approach with a GRU-enhanced policy network for cold-start-aware microservice scheduling in dynamic edge environments.

Findings

Accelerates convergence by 70%

Improves scheduling objective by 50%

Demonstrates high stability across configurations

Abstract

With the rapid growth of IoT devices and their diverse workloads, container-based microservices deployed at edge nodes have become a lightweight and scalable solution. However, existing microservice scheduling algorithms often assume static resource availability, which is unrealistic when multiple containers are assigned to an edge node. Besides, containers suffer from cold-start inefficiencies during early-stage training in currently popular reinforcement learning (RL) algorithms. In this paper, we propose a hybrid learning framework that combines offline imitation learning (IL) with online Soft Actor-Critic (SAC) optimization to enable a cold-start-aware microservice scheduling with dynamic allocation for computing resources. We first formulate a delay-and-energy-aware scheduling problem and construct a rule-based expert to generate demonstration data for behavior cloning. Then, a GRU-enhanced policy network is designed in the policy network to extract the correlation among multiple decisions by separately encoding slow-evolving node states and fast-changing microservice features, and an action selection mechanism is given to speed up the convergence. Extensive experiments show that our method significantly accelerates convergence and achieves superior final performance. Compared with baselines, our algorithm improves the total objective by $50\%$ and convergence speed by $70\%$, and demonstrates the highest stability and robustness across various edge configurations.