PerfEnforce: A Dynamic Scaling Engine for Analytics with Performance Guarantees

TL;DR

PerfEnforce is a dynamic scaling engine for cloud data analytics that optimizes cluster size to meet performance guarantees cost-effectively, using advanced learning methods for decision-making.

Contribution

The paper introduces PerfEnforce, a novel scaling engine that employs perceptron learning to improve cluster scaling decisions for performance guarantees.

Findings

Perceptron learning outperforms feedback control and reinforcement learning in scaling decisions.

PerfEnforce effectively minimizes cost while probabilistically meeting query runtime guarantees.

The approach maintains user session stability during scaling operations.

Abstract

In this paper, we present PerfEnforce, a scaling engine designed to enable cloud providers to sell performance levels for data analytics cloud services. PerfEnforce scales a cluster of virtual machines allocated to a user in a way that minimizes cost while probabilistically meeting the query runtime guarantees offered by a service level agreement. With PerfEnforce, we show how to scale a cluster in a way that minimally disrupts a user's query session. We further show when to scale the cluster using one of three methods: feedback control, reinforcement learning, or perceptron learning. We find that perceptron learning outperforms the other two methods when making cluster scaling decisions.