Algorithmic information and simplicity in statistical physics

TL;DR

This paper explores the relationship between algorithmic information and entropy in statistical physics, demonstrating how to modify universal computers to tighten bounds on average information content.

Contribution

It introduces a method to modify universal computers, removing the constant term in the inequality relating entropy and average algorithmic information.

Findings

Inequality between entropy and average algorithmic information established

Modification of universal computers to eliminate constant term

Tighter bounds improve understanding of information in statistical physics

Abstract

Given a list of $N$ states with probabilities $0<p_1\leq\cdots\leq p_N$, the average conditional algorithmic information $\bar I$ to specify one of these states obeys the inequality $H\leq\bar I<H+O(1)$, where $H=-\sum p_j\log_2p_j$ and $O(1)$ is a computer-dependent constant. We show how any universal computer can be slightly modified in such a way that the inequality becomes $H\leq\bar I<H+1$, thereby eliminating the computer-dependent constant from statistical physics.