How Should We Evaluate Data Deletion in Graph-Based ANN Indexes?

TL;DR

This paper introduces a comprehensive evaluation framework for data deletion in graph-based Approximate Nearest Neighbor Search (ANNS) indexes, addressing a gap in assessing dynamic data support in practical applications.

Contribution

It formalizes and categorizes data deletion methods in graph-based ANNS, proposes evaluation metrics, and demonstrates the framework on Hierarchical Navigable Small World, including a new deletion control method.

Findings

Evaluation framework effectively measures deletion efficiency

Hierarchical Navigable Small World performance analyzed under deletion scenarios

Deletion Control dynamically optimizes deletion method selection

Abstract

Approximate Nearest Neighbor Search (ANNS) has recently gained significant attention due to its many applications, such as Retrieval-Augmented Generation. Such applications require ANNS algorithms that support dynamic data, so the ANNS problem on dynamic data has attracted considerable interest. However, a comprehensive evaluation methodology for data deletion in ANNS has yet to be established. This study proposes an experimental framework and comprehensive evaluation metrics to assess the efficiency of data deletion for ANNS indexes under practical use cases. Specifically, we categorize data deletion methods in graph-based ANNS into three approaches and formalize them mathematically. The performance is assessed in terms of accuracy, query speed, and other relevant metrics. Finally, we apply the proposed evaluation framework to Hierarchical Navigable Small World, one of the state-of-the-art ANNS methods, to analyze the effects of data deletion, and propose Deletion Control, a method which dynamically selects the appropriate deletion method under a required search accuracy.