MonoM: Enhancing Monotonicity in Learned Cardinality Estimators

TL;DR

This paper introduces MonoM, a metric and training framework that enhances the monotonicity and accuracy of learned cardinality estimators in database query optimization.

Contribution

It proposes a novel monotonic training method with a workload generator and regularization, improving both monotonicity adherence and estimation accuracy.

Findings

Monotonic training improves estimator monotonicity.

Regularization reduces overfitting and enhances accuracy.

MonoM metric effectively measures monotonicity adherence.

Abstract

Cardinality estimation is a key component of database query optimization. Recent studies have demonstrated that learned cardinality estimation techniques can surpass traditional methods in accuracy. However, a significant barrier to their adoption in production systems is their tendency to violate fundamental logical principles such as monotonicity. In this paper, we explore how learned models specifically MSCN, a query driven deep learning algorithm can breach monotonicity constraints. To address this, we propose a metric called MonoM, which quantitatively measures how well a cardinality estimator adheres to monotonicity across a given query workload. We also propose a monotonic training framework which includes a workload generator that produces directly comparable queries (one query's predicates are strictly more relaxed than another's, enabling monotonicity inference without actual execution) and a novel regularization term added to the loss function. Experimental results show that our monotonic training algorithm not only enhances monotonicity adherence but also improves cardinality estimation accuracy. This improvement is attributed to the regularization term, which reduces overfitting and improves model generalization.