# ON-NSW: Accelerating High-Dimensional Vector Search on Edge Devices with GPU-Optimized NSW

## TL;DR

This paper introduces ON-NSW, a GPU-optimized version of HNSW for edge devices, improving vector search efficiency and throughput.

## Contribution

ON-NSW is a novel GPU-optimized design of HNSW tailored for edge devices, leveraging parallelism and unified memory architecture.

## Key findings

- ON-NSW achieves up to 1.44× higher throughput than HNSW on NVIDIA Jetson devices.
- The design maintains comparable recall while reducing search latency through warp-level parallelism and synchronization.

## Abstract

The Industrial Internet of Things (IIoT) increasingly relies on vector embeddings for analytics and AI-driven applications such as anomaly detection, predictive maintenance, and sensor fusion. Efficient approximate nearest neighbor search (ANNS) is essential for these workloads. Graph-based methods are among the most representative methods for ANNS. However, most existing graph-based methods, such as Hierarchical Navigable Small World (HNSW), are designed for CPU execution on high-end servers and give little consideration to the unique characteristics of edge devices. In this work, we present ON-NSW, a GPU-optimized design of HNSW optimized for edge devices. ON-NSW employs a flat graph structure derived from HNSW to fully exploit GPU parallelism. In addition, it carefully places HNSW components in the unified memory architecture of NVIDIA Jetson Orin Nano. Also, ON-NSW introduces warp-level parallel neighbor exploration and lightweight synchronization to reduce search latency. Our experimental results on real-world high-dimensional datasets show that ON-NSW achieves up to 1.44× higher throughput than the original HNSW on the NVIDIA Jetson device while maintaining comparable recall. These results demonstrate that ON-NSW provides an effective design for enabling efficient and high-throughput vector search on embedded edge platforms.