p-Adic scaled space filling curve indices for high dimensional data

TL;DR

This paper introduces p-adic Hilbert curves for high-dimensional data indexing, offering a more space-efficient alternative to traditional Hilbert curves, with a method to evaluate data sparsity.

Contribution

It generalizes Hilbert curves to p-adic versions using affine transformations of the p-adic Gray code and develops an efficient scaled indexing method for high-dimensional data.

Findings

p-adic Hilbert curves are less space-consuming than standard Hilbert indices in high dimensions

A measure for local data sparsity is derived and tested

The new indexing method improves efficiency for high-dimensional datasets

Abstract

Space filling curves are widely used in Computer Science. In particular Hilbert curves and their generalisations to higher dimension are used as an indexing method because of their nice locality properties. This article generalises this concept to the systematic construction of p-adic versions of Hilbert curves based on affine transformations of the p-adic Gray code, and develops an efficient scaled indexing method for data taken from high-dimensional spaces based on these new curves, which with increasing dimension is shown to be less space consuming than the optimal standard static Hilbert curve index. A measure is derived which allows to assess the local sparsity of a data set, and is tested on some data.