WebANNS: Fast and Efficient Approximate Nearest Neighbor Search in Web Browsers

TL;DR

WebANNS is a novel in-browser approximate nearest neighbor search engine that significantly improves speed and memory efficiency for web applications, enabling practical AI retrieval tasks directly in browsers.

Contribution

WebANNS introduces a WebAssembly-based engine with lazy loading and heuristic memory reduction, addressing computational, storage, and memory challenges unique to web browsers.

Findings

Achieves up to 743.8x faster query latency than SOTA

Reduces memory usage by up to 39%

Decreases query time from 10 seconds to 10 milliseconds

Abstract

Approximate nearest neighbor search (ANNS) has become vital to modern AI infrastructure, particularly in retrieval-augmented generation (RAG) applications. Numerous in-browser ANNS engines have emerged to seamlessly integrate with popular LLM-based web applications, while addressing privacy protection and challenges of heterogeneous device deployments. However, web browsers present unique challenges for ANNS, including computational limitations, external storage access issues, and memory utilization constraints, which state-of-the-art (SOTA) solutions fail to address comprehensively. We propose WebANNS, a novel ANNS engine specifically designed for web browsers. WebANNS leverages WebAssembly to overcome computational bottlenecks, designs a lazy loading strategy to optimize data retrieval from external storage, and applies a heuristic approach to reduce memory usage. Experiments show that WebANNS is fast and memory efficient, achieving up to $743.8\times$ improvement in 99th percentile query latency over the SOTA engine, while reducing memory usage by up to 39\%. Note that WebANNS decreases query time from 10 seconds to the 10-millisecond range in browsers, making in-browser ANNS practical with user-acceptable latency.