Neuro-Symbolic Query Compiler

TL;DR

This paper introduces QCompiler, a neuro-symbolic query compiler that formalizes complex search queries into precise Abstract Syntax Trees, enhancing retrieval accuracy in RAG systems under resource constraints.

Contribution

It designs a minimal, complete BNF grammar for complex queries and develops a compiler framework to translate queries into executable ASTs, improving retrieval precision.

Findings

Enhanced document retrieval accuracy for complex queries

Effective translation of queries into executable ASTs

Significant improvement in RAG system performance

Abstract

Precise recognition of search intent in Retrieval-Augmented Generation (RAG) systems remains a challenging goal, especially under resource constraints and for complex queries with nested structures and dependencies. This paper presents QCompiler, a neuro-symbolic framework inspired by linguistic grammar rules and compiler design, to bridge this gap. It theoretically designs a minimal yet sufficient Backus-Naur Form (BNF) grammar $G[q]$ to formalize complex queries. Unlike previous methods, this grammar maintains completeness while minimizing redundancy. Based on this, QCompiler includes a Query Expression Translator, a Lexical Syntax Parser, and a Recursive Descent Processor to compile queries into Abstract Syntax Trees (ASTs) for execution. The atomicity of the sub-queries in the leaf nodes ensures more precise document retrieval and response generation, significantly improving the RAG system's ability to address complex queries.