A Framework for dynamically meeting performance objectives on a service mesh

TL;DR

This paper introduces a reinforcement learning-based framework for dynamically managing multiple service mesh objectives such as delay, throughput, and cost, by training an agent in a simulator to optimize control actions.

Contribution

It presents a novel top-down approach to define management objectives and map them onto control actions, enabling simultaneous control and efficient training via simulation.

Findings

Framework effectively manages multiple objectives in a service mesh

Simulation-based training significantly reduces training time

Supports concurrent control actions for complex management goals

Abstract

We present a framework for achieving end-to-end management objectives for multiple services that concurrently execute on a service mesh. We apply reinforcement learning (RL) techniques to train an agent that periodically performs control actions to reallocate resources. We develop and evaluate the framework using a laboratory testbed where we run information and computing services on a service mesh, supported by the Istio and Kubernetes platforms. We investigate different management objectives that include end-to-end delay bounds on service requests, throughput objectives, cost-related objectives, and service differentiation. We compute the control policies on a simulator rather than on the testbed, which speeds up the training time by orders of magnitude for the scenarios we study. Our proposed framework is novel in that it advocates a top-down approach whereby the management objectives are defined first and then mapped onto the available control actions. It allows us to execute several types of control actions simultaneously. By first learning the system model and the operating region from testbed traces, we can train the agent for different management objectives in parallel.