FinSQL: Model-Agnostic LLMs-based Text-to-SQL Framework for Financial Analysis

TL;DR

This paper introduces FinSQL, a model-agnostic LLM-based framework for financial Text-to-SQL tasks, along with a new benchmark dataset BULL, enabling better performance and transferability in financial database querying.

Contribution

The paper provides the first practical financial Text-to-SQL benchmark dataset and proposes a novel LLM-based framework tailored for financial database querying, including prompt design and fine-tuning strategies.

Findings

FinSQL achieves state-of-the-art performance on BULL.

FinSQL improves transfer learning performance by up to 36.64%.

The framework effectively handles wide tables in financial databases.

Abstract

Text-to-SQL, which provides zero-code interface for operating relational databases, has gained much attention in financial analysis; because, financial professionals may not well-skilled in SQL programming. However, until now, there is no practical Text-to-SQL benchmark dataset for financial analysis, and existing Text-to-SQL methods have not considered the unique characteristics of databases in financial applications, such as commonly existing wide tables. To address these issues, we collect a practical Text-to-SQL benchmark dataset and propose a model-agnostic Large Language Model (LLMs)-based Text-to-SQL framework for financial analysis. The benchmark dataset, BULL, is collected from the practical financial analysis business of Hundsun Technologies Inc., including databases for fund, stock, and macro economy. Besides, the proposed LLMs-based Text-to-SQL framework, FinSQL, provides a systematic treatment for financial Text-to-SQL from the perspectives of prompt construction, parameter-efficient fine-tuning and output calibration. Extensive experimental results on BULL demonstrate that FinSQL achieves the state-of-the-art Text-to-SQL performance at a small cost; furthermore, FinSQL can bring up to 36.64% performance improvement in scenarios requiring few-shot cross-database model transfer.