Calculating Kolmogorov Complexity from the Output Frequency Distributions of Small Turing Machines

TL;DR

This paper introduces a new numerical method based on output frequency distributions of small Turing machines to approximate the Kolmogorov complexity of short strings, offering an alternative to traditional compression methods.

Contribution

It presents a novel approach using Turing machine output distributions to estimate Kolmogorov complexity, with extensive analysis and robustness testing for short strings.

Findings

Method accurately estimates complexity of short strings

High stability and robustness demonstrated through statistical analysis

Complementary to traditional compression algorithms

Abstract

Drawing on various notions from theoretical computer science, we present a novel numerical approach, motivated by the notion of algorithmic probability, to the problem of approximating the Kolmogorov-Chaitin complexity of short strings. The method is an alternative to the traditional lossless compression algorithms, which it may complement, the two being serviceable for different string lengths. We provide a thorough analysis for all $\sum_{n=1}^{11} 2^n$ binary strings of length $n<12$ and for most strings of length $12\leq n \leq16$ by running all $\sim 2.5 \times 10^{13}$ Turing machines with 5 states and 2 symbols ($8\times 22^9$ with reduction techniques) using the most standard formalism of Turing machines, used in for example the Busy Beaver problem. We address the question of stability and error estimation, the sensitivity of the continued application of the method for wider coverage and better accuracy, and provide statistical evidence suggesting robustness. As with compression algorithms, this work promises to deliver a range of applications, and to provide insight into the question of complexity calculation of finite (and short) strings.