ListK: Semantic ORDER BY and LIMIT K with Listwise Prompting

TL;DR

ListK introduces a framework that significantly reduces latency in semantic ORDER BY and LIMIT K operations in SQL queries using listwise ranking algorithms, without sacrificing accuracy.

Contribution

The paper presents novel listwise ranking algorithms, including multi-pivot quickselect/sort, and a query optimizer that together improve semantic ORDER BY and LIMIT K performance.

Findings

Halves latency compared to prior methods

Maintains accuracy and NDCG levels

Provides theoretical analysis for parameter tuning

Abstract

Semantic operators abstract large language model (LLM) calls in SQL clauses. It is gaining traction as an easy method to analyze semi-structured, unstructured, and multimodal datasets. While a plethora of recent works optimize various semantic operators, existing methods for semantic ORDER BY (full sort) and LIMIT K (top-K) remain lackluster. Our ListK framework improves the latency of semantic ORDER BY ... LIMIT K at no cost to accuracy. Motivated by the recent advance in fine-tuned listwise rankers, we study several sorting algorithms that best combine partial listwise rankings. These include: 1) deterministic listwise tournament (LTTopK), 2) Las Vegas and embarrassingly parallel listwise multi-pivot quickselect/sort (LMPQSelect, LMPQSort), and 3) a basic Monte Carlo listwise tournament filter (LTFilter). Of these, listwise multi-pivot quickselect/sort are studied here for the first time. The full framework provides a query optimizer for combining the above physical operators based on the target recall to minimize latency. We provide theoretical analysis to easily tune parameters and provide cost estimates for query optimizers. ListK empirically dominates the Pareto frontier, halving latency at virtually no cost to recall and NDCG compared to prior art.