CONCERTO: Complex Query Execution Mechanism-Aware Learned Cost Estimation
Kaixin Zhang, Hongzhi Wang, Kunkai Gu, Ziqi Li, Chunyu Zhao, Yingze, Li, Yu Yan

TL;DR
CONCERTO introduces a novel learned cost estimation framework for complex query execution mechanisms, effectively modeling resource competition and parallelism to improve prediction accuracy in modern DBMSs.
Contribution
It develops a new approach combining GATs and TCNs to accurately predict query performance for complex, parallel, and resource-sharing execution mechanisms.
Findings
CONCERTO outperforms existing prediction methods in accuracy.
It effectively models resource competition among concurrent operators.
The approach adapts to complex execution mechanisms beyond traditional plans.
Abstract
With the growing demand for massive data analysis, many DBMSs have adopted complex underlying query execution mechanisms, including vectorized operators, parallel execution, and dynamic pipeline modifications. However, there remains a lack of targeted Query Performance Prediction (QPP) methods for these complex execution mechanisms and their interactions, as most existing approaches focus on traditional tree-shaped query plans and static serial executors. To address this challenge, this paper proposes CONCERTO, a Complex query executiON meChanism-awaE leaRned cosT estimatiOn method. CONCERTO first establishes independent resource cost models for each physical operator. It then constructs a Directed Acyclic Graph (DAG) consisting of a dataflow tree backbone and resource competition relationships among concurrent operators. After calibrating the cost impact of parallel operator execution…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsData Management and Algorithms · Algorithms and Data Compression · Caching and Content Delivery
MethodsSoftmax · Attention Is All You Need · Focus · ADaptive gradient method with the OPTimal convergence rate · Graph Attention Network
