Glue: Adaptively Merging Single Table Cardinality to Estimate Join Query Size

TL;DR

This paper introduces Glue, a flexible framework that combines single-table cardinality estimates to accurately and efficiently estimate join query sizes across complex schemas, adaptable to various performance needs.

Contribution

Glue provides a general, decoupled approach to merge single-table cardinality estimates for join size estimation, supporting any existing method and complex schemas.

Findings

Supports any existing CardEst method for single tables

Adapts to different scenarios like OLTP and OLAP

Seamlessly integrates into plan search and supports distinct count estimation

Abstract

Cardinality estimation (CardEst), a central component of the query optimizer, plays a significant role in generating high-quality query plans in DBMS. The CardEst problem has been extensively studied in the last several decades, using both traditional and ML-enhanced methods. Whereas, the hardest problem in CardEst, i.e., how to estimate the join query size on multiple tables, has not been extensively solved. Current methods either reply on independence assumptions or apply techniques with heavy burden, whose performance is still far from satisfactory. Even worse, existing CardEst methods are often designed to optimize one goal, i.e., inference speed or estimation accuracy, which can not adapt to different occasions. In this paper, we propose a very general framework, called Glue, to tackle with these challenges. Its key idea is to elegantly decouple the correlations across different tables and losslessly merge single table CardEst results to estimate the join query size. Glue supports obtaining the single table-wise CardEst results using any existing CardEst method and can process any complex join schema. Therefore, it easily adapts to different scenarios having different performance requirements, i.e., OLTP with fast estimation time or OLAP with high estimation accuracy. Meanwhile, we show that Glue can be seamlessly integrated into the plan search process and is able to support counting distinct number of values. All these properties exhibit the potential advances of deploying Glue in real-world DBMS.