flexvec: SQL Vector Retrieval with Programmatic Embedding Modulation

TL;DR

flexvec introduces a programmable retrieval kernel that allows dynamic manipulation of embeddings and scores at query time, enabling flexible and efficient vector retrieval within SQL interfaces for large datasets.

Contribution

The paper presents flexvec, a novel retrieval kernel with programmable embedding modulation, integrated into SQL, allowing composable and efficient vector retrieval operations.

Findings

Operates on 240,000 chunks in 19 ms on a desktop CPU.

Handles 1 million chunks in 82 ms without approximate indexing.

Enables flexible, programmable retrieval pipelines for large-scale data.

Abstract

As AI agents become the primary consumers of retrieval APIs, there is an opportunity to expose more of the retrieval pipeline to the caller. flexvec is a retrieval kernel that exposes the embedding matrix and score array as a programmable surface, allowing arithmetic operations on both before selection. We refer to composing operations on this surface at query time as Programmatic Embedding Modulation (PEM). This paper describes a set of such operations and integrates them into a SQL interface via a query materializer that facilitates composable query primitives. On a production corpus of 240,000 chunks, three composed modulations execute in 19 ms end-to-end on a desktop CPU without approximate indexing. At one million chunks, the same operations execute in 82 ms.