Semantic Caching for OLAP via LLM-Based Query Canonicalization (Extended Version)

TL;DR

This paper presents a middleware cache for OLAP workloads that uses LLM-based query canonicalization to improve reuse across diverse BI tools and interfaces, achieving high hit rates with strict correctness guarantees.

Contribution

It introduces the OLAP Intent Signature, a unified canonicalization method for SQL and NL queries, enabling safe and effective caching with derivations to extend coverage.

Findings

82% cache hit rate on benchmark queries

Zero false hits with strict validation

Derivations double hit rate on hierarchical queries

Abstract

Analytical workloads exhibit substantial semantic repetition, yet most production caches key entries by SQL surface form (text or AST), fragmenting reuse across BI tools, notebooks, and NL interfaces. We introduce a safety-first middleware cache for dashboard-style OLAP over star schemas that canonicalizes both SQL and NL into a unified key space -- the OLAP Intent Signature -- capturing measures, grouping levels, filters, and time windows. Reuse requires exact intent matches under strict schema validation and confidence-gated NL acceptance; two correctness-preserving derivations (roll-up, filter-down) extend coverage without approximate matching. Across TPC-DS, SSB, and NYC TLC (1,395 queries), we achieve 82% hit rate versus 28% (text) and 56% (AST) with zero false hits; derivations double hit rate on hierarchical queries.