Multi-agent Reinforcement Learning-based In-place Scaling Engine for Edge-cloud Systems

TL;DR

This paper introduces MARLISE, a multi-agent reinforcement learning system for in-place resource scaling in edge-cloud systems, improving efficiency and responsiveness over traditional static methods.

Contribution

It presents a novel multi-agent reinforcement learning approach using DQN and PPO for dynamic in-place scaling in edge-cloud environments.

Findings

MARLISE outperforms heuristic methods in resource management.

It maintains low microservice response times under dynamic workloads.

MARLISE achieves higher resource efficiency.

Abstract

Modern edge-cloud systems face challenges in efficiently scaling resources to handle dynamic and unpredictable workloads. Traditional scaling approaches typically rely on static thresholds and predefined rules, which are often inadequate for optimizing resource utilization and maintaining performance in distributed and dynamic environments. This inefficiency hinders the adaptability and performance required in edge-cloud infrastructures, which can only be achieved through the newly proposed in-place scaling. To address this problem, we propose the Multi-Agent Reinforcement Learning-based In-place Scaling Engine (MARLISE) that enables seamless, dynamic, reactive control with in-place resource scaling. We develop our solution using two Deep Reinforcement Learning algorithms: Deep Q-Network (DQN), and Proximal Policy Optimization (PPO). We analyze each version of the proposed MARLISE solution using dynamic workloads, demonstrating their ability to ensure low response times of microservices and scalability. Our results show that MARLISE-based approaches outperform heuristic method in managing resource elasticity while maintaining microservice response times and achieving higher resource efficiency.