DIVER: A Robust Text-to-SQL System with Dynamic Interactive Value Linking and Evidence Reasoning

TL;DR

DIVER is a robust Text-to-SQL system that automates evidence reasoning and dynamic value linking, significantly improving accuracy and robustness in database query tasks without requiring expert input.

Contribution

It introduces a novel automated evidence reasoning approach with dynamic interactive value linking, enhancing robustness and accuracy of Text-to-SQL models in complex, real-world scenarios.

Findings

Performance improved by up to 10.82% in execution accuracy

Enhanced robustness in schema and value linking

Significant gains in handling large-scale, dynamic database values

Abstract

In the era of large language models, Text-to-SQL, as a natural language interface for databases, is playing an increasingly important role. The sota Text-to-SQL models have achieved impressive accuracy, but their performance critically relies on expert-written evidence, which typically clarifies schema and value linking that existing models struggle to identify. Such limitations stem from the ambiguity of user queries and, more importantly, the complexity of comprehending large-scale and dynamic database values. Consequently, in real-world scenarios where expert assistance is unavailable, existing methods suffer a severe performance collapse, with execution accuracy dropping by over 10%. This underscores their lack of robustness. To address this, we propose DIVER, a robust system that automates evidence reasoning with dynamic interactive value linking. It leverages a compatible toolbox containing diverse tools to probe the database. Then, restricted by a structured workspace (CoTF, Chain of Thoughts and Facts), it reflects based on probe results and selects a new tool for next round of probing. Through this automatically iterative process, DIVER identifies schema and value linking missed by existing methods. Based on these accurate linkings, DIVER is able to infer correct usage of SQL functions and formulas and generate high-quality evidence, achieving robust Text-to-SQL without expert assistance. Extensive experiments demonstrate that: 1) The DIVER system significantly enhances the robustness of various Text-to-SQL models, improving performance by up to 10.82% in Execution Accuracy (EX) and 16.09% in Valid Efficiency Score (VES). 2) Our dynamic interactive value linking significantly improves the robustness of existing systems and the accuracy of schema and value linking, especially when confronted with challenges posed by large-scale, dynamic database values.