Symmetry of Information: A Closer Look

TL;DR

This paper revisits the symmetry of information in algorithmic information theory, providing a simpler proof of a refined relation between mutual information of strings, with implications for randomness and complexity.

Contribution

It offers a simpler proof of a tighter symmetry-of-information relation for strings with simple complexity, extending results to randomness and complexity notions.

Findings

Simpler proof of symmetry-of-information for simple complexity strings

Tighter bounds on mutual information between strings

Extension of results to randomness and complexity notions

Abstract

Symmetry of information establishes a relation between the information that x has about y (denoted I(x : y)) and the information that y has about x (denoted I(y : x)). In classical information theory, the two are exactly equal, but in algorithmical information theory, there is a small excess quantity of information that differentiates the two terms, caused by the necessity of packaging information in a way that makes it accessible to algorithms. It was shown in [Zim11] that in the case of strings with simple complexity (that is the Kolmogorov complexity of their Kolmogorov complexity is small), the relevant information can be packed in a very economical way, which leads to a tighter relation between I(x : y) and I(y : x) than the one provided in the classical symmetry-of-information theorem of Kolmogorov and Levin. We give here a simpler proof of this result, using a suggestion of Alexander Shen. This result implies a van Lambalgen- type theorem for finite strings and plain complexity: If x is c-random and y is c-random relative to x, then xy is O(c)-random. We show that a similar result holds for prefix-free complexity and weak-K-randomness.