Querying Databases with Function Calling
Connor Shorten, Charles Pierse, Thomas Benjamin Smith, Karel, D'Oosterlinck, Tuana Celik, Erika Cardenas, Leonie Monigatti, Mohd Shukri, Hasan, Edward Schmuhl, Daniel Williams, Aravind Kesiraju, Bob van Luijt

TL;DR
This paper explores how Large Language Models can effectively query databases using Function Calling, demonstrating high accuracy and robustness across models and use cases, and providing a unified tool definition for database interactions.
Contribution
It introduces a unified tool definition for database querying with Function Calling and evaluates its effectiveness across multiple LLMs and synthetic database scenarios.
Findings
Claude 3.5 Sonnet achieves 74.3% Exact Match score.
Models excel at boolean property operators but struggle with text filters.
Higher performing models like GPT-4o show robust results across use cases.
Abstract
The capabilities of Large Language Models (LLMs) are rapidly accelerating largely thanks to their integration with external tools. Querying databases is among the most effective of these integrations, enabling LLMs to access private or continually updating data. While Function Calling is the most common method for interfacing external tools to LLMs, its application to database querying as a tool has been underexplored. We propose a tool definition for database querying that unifies accessing data with search queries, filters, or a combination both, as well as transforming results with aggregation and groupby operators. To evaluate its effectiveness, we conduct a study with 8 LLMs spanning 5 model families. We present a novel pipeline adapting the Gorilla LLM framework to create synthetic database schemas and queries. We primarily evaluate the models with the Exact Match of predicted and…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsData Management and Algorithms · Advanced Database Systems and Queries · Graph Theory and Algorithms
