Text2Cypher: Data Pruning using Hard Example Selection

TL;DR

This paper introduces five hard-example selection techniques to prune the Text2Cypher dataset, significantly reducing training costs and time while maintaining performance in natural language to Cypher query translation.

Contribution

The paper proposes novel hard-example selection methods for dataset pruning in Text2Cypher, improving training efficiency without sacrificing accuracy.

Findings

Halves training time and costs

Maintains performance with high-quality dataset pruning

Demonstrates cost-effectiveness of hard-example selection

Abstract

Database query languages such as SQL for relational databases and Cypher for graph databases have been widely adopted. Recent advancements in large language models (LLMs) enable natural language interactions with databases through models like Text2SQL and Text2Cypher. Fine-tuning these models typically requires large, diverse datasets containing non-trivial examples. However, as dataset size increases, the cost of fine-tuning also rises. This makes smaller, high-quality datasets essential for reducing costs for the same or better performance. In this paper, we propose five hard-example selection techniques for pruning the Text2Cypher dataset, aiming to preserve or improve performance while reducing resource usage. Our results show that these hard-example selection approaches can halve training time and costs with minimal impact on performance, and demonstrates that hard-example selection provides a cost-effective solution.