Vector Search for the Future: From Memory-Resident, Static Heterogeneous Storage, to Cloud-Native Architectures

TL;DR

This paper reviews the evolution of vector search architectures from in-memory to cloud-native, highlighting how storage innovations address scalability, cost, and performance challenges in large-scale multimodal data retrieval.

Contribution

It provides a systematic overview of vector search techniques across different storage architectures and discusses future directions for cloud-native, trillion-scale retrieval systems.

Findings

Memory-resident VS methods are fast but limited in scalability.

Heterogeneous storage architectures improve scalability and cost-efficiency.

Emerging cloud-native systems enable seamless scalability for trillion-scale data.

Abstract

Vector search (VS) has become a fundamental component in multimodal data management, enabling core functionalities such as image, video, and code retrieval. As vector data scales rapidly, VS faces growing challenges in balancing search, latency, scalability, and cost. The evolution of VS has been closely driven by changes in storage architecture. Early VS methods rely on all-in-memory designs for low latency, but scalability is constrained by memory capacity and cost. To address this, recent research has adopted heterogeneous architectures that offload space-intensive vectors and index structures to SSDs, while exploiting block locality and I/O-efficient strategies to maintain high search performance at billion scale. Looking ahead, the increasing demand for trillion-scale vector retrieval and cloud-native elasticity is driving a further shift toward memory-SSD-object storage architectures, which enable cost-efficient data tiering and seamless scalability. In this tutorial, we review the evolution of VS techniques from a storage-architecture perspective. We first review memory-resident methods, covering classical IVF, hash, quantization, and graph-based designs. We then present a systematic overview of heterogeneous storage VS techniques, including their index designs, block-level layouts, query strategies, and update mechanisms. Finally, we examine emerging cloud-native systems and highlight open research opportunities for future large-scale vector retrieval systems.