Nonapproximablity of the Normalized Information Distance

TL;DR

This paper proves that the theoretical normalized information distance (NID), based on Kolmogorov complexity, cannot be approximated from above or below, highlighting fundamental limitations in its computability.

Contribution

It establishes that the NID is neither upper nor lower semicomputable within any reasonable bounds, clarifying its theoretical complexity properties.

Findings

NID is not upper semicomputable.

NID is not lower semicomputable.

Implications for practical approximation methods.

Abstract

Normalized information distance (NID) uses the theoretical notion of Kolmogorov complexity, which for practical purposes is approximated by the length of the compressed version of the file involved, using a real-world compression program. This practical application is called `normalized compression distance' and it is trivially computable. It is a parameter-free similarity measure based on compression, and is used in pattern recognition, data mining, phylogeny, clustering, and classification. The complexity properties of its theoretical precursor, the NID, have been open. We show that the NID is neither upper semicomputable nor lower semicomputable up to any reasonable precision.