A Practical Theory of Generalization in Selectivity Learning

TL;DR

This paper develops a theoretical framework for query-driven selectivity learning, demonstrating learnability and establishing out-of-distribution generalization bounds, and empirically improves model performance on OOD queries.

Contribution

It introduces a relaxed theoretical model for selectivity predictors and provides strategies to enhance out-of-distribution generalization in query-driven learning.

Findings

Selectivity predictors induced by signed measures are learnable.

Proposed techniques significantly improve OOD generalization in practice.

Models maintain high in-distribution performance while improving OOD accuracy.

Abstract

Query-driven machine learning models have emerged as a promising estimation technique for query selectivities. Yet, surprisingly little is known about the efficacy of these techniques from a theoretical perspective, as there exist substantial gaps between practical solutions and state-of-the-art (SOTA) theory based on the Probably Approximately Correct (PAC) learning framework. In this paper, we aim to bridge the gaps between theory and practice. First, we demonstrate that selectivity predictors induced by signed measures are learnable, which relaxes the reliance on probability measures in SOTA theory. More importantly, beyond the PAC learning framework (which only allows us to characterize how the model behaves when both training and test workloads are drawn from the same distribution), we establish, under mild assumptions, that selectivity predictors from this class exhibit favorable out-of-distribution (OOD) generalization error bounds. These theoretical advances provide us with a better understanding of both the in-distribution and OOD generalization capabilities of query-driven selectivity learning, and facilitate the design of two general strategies to improve OOD generalization for existing query-driven selectivity models. We empirically verify that our techniques help query-driven selectivity models generalize significantly better to OOD queries both in terms of prediction accuracy and query latency performance, while maintaining their superior in-distribution generalization performance.