A geometric framework for modelling similarity search

TL;DR

This paper introduces a geometric framework for modeling similarity search in high-dimensional data spaces, integrating metric geometry concepts to analyze complexity, indexability, and the curse of dimensionality.

Contribution

It proposes a novel geometric framework based on metric geometry concepts to better understand and analyze similarity search challenges in large, high-dimensional datasets.

Findings

Provides a geometric perspective on similarity workloads

Analyzes the curse of dimensionality using measure concentration

Bridges database research with metric geometry techniques

Abstract

The aim of this paper is to propose a geometric framework for modelling similarity search in large and multidimensional data spaces of general nature, which seems to be flexible enough to address such issues as analysis of complexity, indexability, and the `curse of dimensionality.' Such a framework is provided by the concept of the so-called similarity workload, which is a probability metric space $\Omega$ (query domain) with a distinguished finite subspace $X$ (dataset), together with an assembly of concepts, techniques, and results from metric geometry. They include such notions as metric transform, $\e$-entropy, and the phenomenon of concentration of measure on high-dimensional structures. In particular, we discuss the relevance of the latter to understanding the curse of dimensionality. As some of those concepts and techniques are being currently reinvented by the database community, it seems desirable to try and bridge the gap between database research and the relevant work already done in geometry and analysis.