UpANNS: Enhancing Billion-Scale ANNS Efficiency with Real-World PIM Architecture

TL;DR

UpANNS introduces a PIM-based framework that significantly accelerates billion-scale ANNS, outperforming CPU solutions and matching GPU performance with higher energy efficiency, suitable for real-time AI applications.

Contribution

The paper presents UpANNS, a novel PIM architecture-aware framework with four innovations that substantially improve efficiency and scalability of billion-scale ANNS.

Findings

4.3x higher QPS than CPU-based Faiss

Matches GPU performance with 2.3x greater energy efficiency

Near-linear scalability for large datasets

Abstract

Approximate Nearest Neighbor Search (ANNS) is a critical component of modern AI systems, such as recommendation engines and retrieval-augmented large language models (RAG-LLMs). However, scaling ANNS to billion-entry datasets exposes critical inefficiencies: CPU-based solutions are bottlenecked by memory bandwidth limitations, while GPU implementations underutilize hardware resources, leading to suboptimal performance and energy consumption. To address these challenges, we introduce \emph{UpANNS}, a novel framework leveraging Processing-in-Memory (PIM) architecture to accelerate billion-scale ANNS. UpANNS integrates four key innovations, including 1) architecture-aware data placement to minimize latency through workload balancing, 2) dynamic resource management for optimal PIM utilization, 3) co-occurrence optimized encoding to reduce redundant computations, and 4) an early-pruning strategy for efficient top-k selection. Evaluation on commercial UPMEM hardware demonstrates that UpANNS achieves 4.3x higher QPS than CPU-based Faiss, while matching GPU performance with 2.3x greater energy efficiency. Its near-linear scalability ensures practicality for growing datasets, making it ideal for applications like real-time LLM serving and large-scale retrieval systems.