Tempo: Robust and Self-Tuning Resource Management in Multi-tenant Parallel Databases

TL;DR

Tempo is a framework that simplifies, self-tunes, and robustly manages resources in multi-tenant databases, enabling direct specification of performance goals and automatic optimization of resource configurations.

Contribution

It introduces a novel framework that allows declarative performance objectives and provides theoretical robustness guarantees for resource management in multi-tenant databases.

Findings

Significant improvements in meeting performance objectives.

Robustness guarantees backed by theoretical foundations.

Effective on real-world production workloads.

Abstract

Multi-tenant database systems have a component called the Resource Manager, or RM that is responsible for allocating resources to tenants. RMs today do not provide direct support for performance objectives such as: "Average job response time of tenant A must be less than two minutes", or "No more than 5% of tenant B's jobs can miss the deadline of 1 hour." Thus, DBAs have to tinker with the RM's low-level configuration settings to meet such objectives. We propose a framework called Tempo that brings simplicity, self-tuning, and robustness to existing RMs. Tempo provides a simple interface for DBAs to specify performance objectives declaratively, and optimizes the RM configuration settings to meet these objectives. Tempo has a solid theoretical foundation which gives key robustness guarantees. We report experiments done on Tempo using production traces of data-processing workloads from companies such as Facebook and Cloudera. These experiments demonstrate significant improvements in meeting desired performance objectives over RM configuration settings specified by human experts.