RubikSQL: Lifelong Learning Agentic Knowledge Base as an Industrial NL2SQL System

TL;DR

RubikSQL is an innovative lifelong learning NL2SQL system that maintains and refines a knowledge base to handle real-world enterprise queries, achieving state-of-the-art results and introducing a new benchmark for industrial scenarios.

Contribution

It introduces RubikSQL, a lifelong learning approach for NL2SQL that combines knowledge base maintenance, rule mining, and multi-agent workflows, along with a new industrial NL2SQL benchmark.

Findings

Achieves state-of-the-art performance on KaggleDBQA and BIRD Mini-Dev datasets.

Develops a comprehensive KB building and refinement process.

Provides the RubikBench benchmark for industrial NL2SQL scenarios.

Abstract

We present RubikSQL, a novel NL2SQL system designed to address key challenges in real-world enterprise-level NL2SQL, such as implicit intents and domain-specific terminology. RubikSQL frames NL2SQL as a lifelong learning task, demanding both Knowledge Base (KB) maintenance and SQL generation. RubikSQL systematically builds and refines its KB through techniques including database profiling, structured information extraction, agentic rule mining, and Chain-of-Thought (CoT)-enhanced SQL profiling. RubikSQL then employs a multi-agent workflow to leverage this curated KB, generating accurate SQLs. RubikSQL achieves SOTA performance on both the KaggleDBQA and BIRD Mini-Dev datasets. Finally, we release the RubikBench benchmark, a new benchmark specifically designed to capture vital traits of industrial NL2SQL scenarios, providing a valuable resource for future research.