Precise Expression for the Algorithmic Information Distance

TL;DR

This paper proves that the information distance between two objects is equal to the maximum of their conditional Kolmogorov complexities up to an additive constant, confirming a conjecture and extending the result to small sets.

Contribution

It confirms that the information distance equals the maximum of conditional complexities with optimal precision and extends this to small sets, resolving subtle definitional issues.

Findings

Distance equals max(K(x|y),K(y|x)) up to O(1) for strings of equal length

Triangle inequality and characterization hold with optimal precision for such strings

Shortest program for small sets is bounded by max complexity plus O(1)

Abstract

We consider the notion of information distance between two objects $x$ and $y$ introduced by Bennett, G\'acs, Li, Vit\'anyi, and Zurek in 1998 as the minimal length of a program that computes $x$ from $y$ as well as computing $y$ from $x$. In this paper, it was proven that the distance is equal to $\max (K(x|y),K(y|x))$ up to additive logarithmic terms, and it was conjectured that this could not be improved to $O(1)$ precision. We revisit subtle issues in the definition and prove this conjecture. We show that if the distance is at least logarithmic in the length, then this equality does hold with $O(1)$ precision for strings of equal length. Thus for such strings, both the triangle inequality and the characterization hold with optimal precision. Finally, we extend the result to sets $S$ of bounded size. We show that for each constant~$s$, the shortest program that prints an $s$-element set $S \subseteq \{0,1\}^n$ given any of its elements, has length at most $\max_{w \in S} K(S|w) + O(1)$, provided this maximum is at least logarithmic in~$n$.