Learned Cardinalities: Estimating Correlated Joins with Deep Learning

TL;DR

This paper introduces MSCN, a deep learning model that improves the accuracy of cardinality estimation in query optimization by capturing complex correlations and addressing sampling limitations.

Contribution

The paper presents MSCN, a novel multi-set convolutional network that models relational query plans and enhances cardinality estimation accuracy over traditional sampling methods.

Findings

Deep learning significantly improves cardinality estimation quality.

MSCN effectively captures join-crossing correlations.

The approach outperforms existing sampling-based methods.

Abstract

We describe a new deep learning approach to cardinality estimation. MSCN is a multi-set convolutional network, tailored to representing relational query plans, that employs set semantics to capture query features and true cardinalities. MSCN builds on sampling-based estimation, addressing its weaknesses when no sampled tuples qualify a predicate, and in capturing join-crossing correlations. Our evaluation of MSCN using a real-world dataset shows that deep learning significantly enhances the quality of cardinality estimation, which is the core problem in query optimization.