DRPC: Distributed Reinforcement Learning Approach for Scalable Resource Provisioning in Container-based Clusters

TL;DR

This paper introduces DRPC, a distributed reinforcement learning method for scalable resource provisioning in container-based microservice clusters, improving response times and reducing failures.

Contribution

It presents a novel decentralized autoscaling approach using deep reinforcement learning to enhance scalability and efficiency in microservice environments.

Findings

Reduces average response time by 15%

Decreases failed requests by 24%

Demonstrates scalability improvements in large-scale clusters

Abstract

Microservices have transformed monolithic applications into lightweight, self-contained, and isolated application components, establishing themselves as a dominant paradigm for application development and deployment in public clouds such as Google and Alibaba. Autoscaling emerges as an efficient strategy for managing resources allocated to microservices' replicas. However, the dynamic and intricate dependencies within microservice chains present challenges to the effective management of scaled microservices. Additionally, the centralized autoscaling approach can encounter scalability issues, especially in the management of large-scale microservice-based clusters. To address these challenges and enhance scalability, we propose an innovative distributed resource provisioning approach for microservices based on the Twin Delayed Deep Deterministic Policy Gradient algorithm. This approach enables effective autoscaling decisions and decentralizes responsibilities from a central node to distributed nodes. Comparative results with state-of-the-art approaches, obtained from a realistic testbed and traces, indicate that our approach reduces the average response time by 15% and the number of failed requests by 24%, validating improved scalability as the number of requests increases.