Towards Accurate and Efficient Document Analytics with Large Language Models

TL;DR

ZenDB is a system that uses semantic structures in templatized unstructured documents combined with LLMs to enable accurate, efficient, and cost-effective ad-hoc SQL queries on large document collections.

Contribution

The paper introduces ZenDB, a novel system that leverages semantic hierarchical structures in templatized documents to improve query accuracy and reduce costs compared to existing LLM and RAG-based methods.

Findings

Achieves up to 30% cost savings over LLM baselines.

Surpasses RAG baselines by 61% in precision and 80% in recall.

Maintains or improves accuracy while reducing costs.

Abstract

Unstructured data formats account for over 80% of the data currently stored, and extracting value from such formats remains a considerable challenge. In particular, current approaches for managing unstructured documents do not support ad-hoc analytical queries on document collections. Moreover, Large Language Models (LLMs) directly applied to the documents themselves, or on portions of documents through a process of Retrieval-Augmented Generation (RAG), fail to provide high accuracy query results, and in the LLM-only case, additionally incur high costs. Since many unstructured documents in a collection often follow similar templates that impart a common semantic structure, we introduce ZenDB, a document analytics system that leverages this semantic structure, coupled with LLMs, to answer ad-hoc SQL queries on document collections. ZenDB efficiently extracts semantic hierarchical structures from such templatized documents, and introduces a novel query engine that leverages these structures for accurate and cost-effective query execution. Users can impose a schema on their documents, and query it, all via SQL. Extensive experiments on three real-world document collections demonstrate ZenDB's benefits, achieving up to 30% cost savings compared to LLM-based baselines, while maintaining or improving accuracy, and surpassing RAG-based baselines by up to 61% in precision and 80% in recall, at a marginally higher cost.