Extending Confidence-Based Text2Cypher with Grammar and Schema Aware Filtering

TL;DR

This paper enhances confidence-based inference for Text2Cypher by incorporating grammar and schema constraints at test time, improving query validity and execution accuracy.

Contribution

It introduces a sequential filtering framework that applies structural constraints during inference, demonstrating improved reliability in Text2Cypher generation.

Findings

Grammar filtering improves syntactic validity.

Schema-aware filtering enhances execution correctness.

Stronger filtering reduces coverage and increases empty outputs.

Abstract

Large language models (LLMs) allow users to query databases using natural language by translating questions into executable queries. Despite strong progress on tasks such as Text2SQL, Text2SPARQL, and Text2Cypher, most existing methods focus on better prompting, fine-tuning, or iterative refinement. However, they often do not explicitly enforce structural constraints, such as syntactic validity and schema consistency. This can reduce reliability, since generated queries must satisfy both syntax rules and database schema constraints to be executable. In this work, we study how structured constraints can be used in test-time inference for Text2Cypher. We focus on post-generation validation to improve query correctness. We extend a confidence-based inference framework with a sequential filtering process that combines confidence scoring, grammar validation, and schema constraints before final aggregation. This lets us analyze how different constraint types affect generated queries. Our experiments with two instruction-tuned models show that grammar-based filtering improves syntactic validity. Schema-aware filtering further improves execution quality by enforcing consistency with the database structure. However, stronger filtering also increases the number of empty predictions and reduces execution coverage. Overall, we show that adding simple structural checks at test time improves the reliability of Text2Cypher generation, and we provide a clearer view of how syntax and schema constraints contribute differently.