Properties of Algorithmic Information Distance

TL;DR

This paper explores the theoretical properties of the un-normalized algorithmic information distance based on Kolmogorov complexity, revealing its metric nature and embedding capabilities, and setting the stage for future research.

Contribution

It investigates the metric and embedding properties of the un-normalized algorithmic information distance, providing new insights and frameworks for understanding its structure.

Findings

Many infinite-dimensional spaces cannot be isometrically embedded into the space of finite strings with $d_K$

The distance $d_K$ is not Euclidean, but finite Euclidean sets can be embedded into it

Frameworks and open problems for future research are developed

Abstract

The domain-independent universal Normalized Information Distance based on Kolmogorov complexity has been (in approximate form) successfully applied to a variety of difficult clustering problems. In this paper we investigate theoretical properties of the un-normalized algorithmic information distance $d_K$. The main question we are asking in this work is what properties this curious distance has, besides being a metric. We show that many (in)finite-dimensional spaces can(not) be isometrically scale-embedded into the space of finite strings with metric $d_K$. We also show that $d_K$ is not an Euclidean distance, but any finite set of points in Euclidean space can be scale-embedded into $(\{0,1\}^*,d_K)$. A major contribution is the development of the necessary framework and tools for finding more (interesting) properties of $d_K$ in future, and to state several open problems.