Cortex AISQL: A Production SQL Engine for Unstructured Data

TL;DR

Snowflake's Cortex AISQL is a production SQL engine that seamlessly integrates semantic reasoning with relational queries, optimizing for efficiency and scalability in unstructured data analysis.

Contribution

The paper introduces three novel techniques—AI-aware query optimization, adaptive model cascades, and semantic join rewriting—that significantly improve performance of semantic operations in production.

Findings

Achieves 2-8× speedups through AI-aware query optimization.

Reduces inference costs by 2-6× with adaptive model cascades.

Speeds up join operations by 15-70× via reformulation as classification.

Abstract

Snowflake's Cortex AISQL is a production SQL engine that integrates native semantic operations directly into SQL. This integration allows users to write declarative queries that combine relational operations with semantic reasoning, enabling them to query both structured and unstructured data effortlessly. However, making semantic operations efficient at production scale poses fundamental challenges. Semantic operations are more expensive than traditional SQL operations, possess distinct latency and throughput characteristics, and their cost and selectivity are unknown during query compilation. Furthermore, existing query engines are not designed to optimize semantic operations. The AISQL query execution engine addresses these challenges through three novel techniques informed by production deployment data from Snowflake customers. First, AI-aware query optimization treats AI inference cost as a first-class optimization objective, reasoning about large language model (LLM) cost directly during query planning to achieve 2-8$\times$ speedups. Second, adaptive model cascades reduce inference costs by routing most rows through a fast proxy model while escalating uncertain cases to a powerful oracle model, achieving 2-6$\times$ speedups while maintaining 90-95% of oracle model quality. Third, semantic join query rewriting lowers the quadratic time complexity of join operations to linear through reformulation as multi-label classification tasks, achieving 15-70$\times$ speedups with often improved prediction quality. AISQL is deployed in production at Snowflake, where it powers diverse customer workloads across analytics, search, and content understanding.