High-Dimensional Bayesian Optimization with Multi-Task Learning for RocksDB

TL;DR

This paper introduces a multi-task Bayesian optimization approach with dimensionality reduction to efficiently tune RocksDB parameters, significantly improving throughput with fewer optimization steps.

Contribution

It presents a novel multi-task Bayesian optimization method with manual parameter grouping for high-dimensional tuning problems like RocksDB.

Findings

Achieved 1.3x throughput improvement over baseline.

Converged in ten steps versus fifty for other methods.

Effectively handled high-dimensional parameter space.

Abstract

RocksDB is a general-purpose embedded key-value store used in multiple different settings. Its versatility comes at the cost of complex tuning configurations. This paper investigates maximizing the throughput of RocksDB IO operations by auto-tuning ten parameters of varying ranges. Off-the-shelf optimizers struggle with high-dimensional problem spaces and require a large number of training samples. We propose two techniques to tackle this problem: multi-task modeling and dimensionality reduction through a manual grouping of parameters. By incorporating adjacent optimization in the model, the model converged faster and found complicated settings that other tuners could not find. This approach had an additional computational complexity overhead, which we mitigated by manually assigning parameters to each sub-goal through our knowledge of RocksDB. The model is then incorporated in a standard Bayesian Optimization loop to find parameters that maximize RocksDB's IO throughput. Our method achieved x1.3 improvement when benchmarked against a simulation of Facebook's social graph traffic, and converged in ten optimization steps compared to other state-of-the-art methods that required fifty steps.