Adversarial Query Synthesis via Bayesian Optimization

TL;DR

This paper introduces a Bayesian optimization method to automatically generate challenging benchmark queries for database systems, reducing manual effort and improving the quality of benchmarks.

Contribution

The paper presents a novel Bayesian optimization approach for automatic query synthesis, enhancing benchmark difficulty with less manual intervention.

Findings

Queries with over twice the optimization headroom

Significant reduction in manual benchmark creation effort

Improved benchmark difficulty over existing methods

Abstract

Benchmark workloads are extremely important to the database management research community, especially as more machine learning components are integrated into database systems. Here, we propose a Bayesian optimization technique to automatically search for difficult benchmark queries, significantly reducing the amount of manual effort usually required. In preliminary experiments, we show that our approach can generate queries with more than double the optimization headroom compared to existing benchmarks.