Undecidability, entropy and information loss in computations of classical physical systems

TL;DR

This paper explores how undecidability and finite information capacity in classical physical systems lead to inevitable entropy increase and information loss, providing an information-theoretic perspective on the second law of thermodynamics.

Contribution

It demonstrates that finite information capacity causes entropy to increase and information to be lost in classical systems, linking undecidability to thermodynamic irreversibility.

Findings

Entropy always increases or information is lost in finite-capacity systems.

Information loss occurs both forward and backward in time under Hamiltonian dynamics.

Finite information capacity causes probability distributions to be incompressible, leading to entropy increase.

Abstract

We investigate how undecidability enters into computations of classical physical systems and contributes to the increase of entropy and loss of information. In actual computation with finite bit of information capacity we accept inconsistency to avoid undecidability, which in turn affects entropy of the system. We apply the Shannon entropy to the discretized Liouvillian system. It is shown that for any finite bit of information capacity information is always lost or the entropy always increases for the probability density following Hamiltonian dynamics, both in time forward and time backward direction, thus showing information theoretical version of second law of thermodynamics. This is due to the finiteness of information capacity and incompressibility of probability distribution in Liouville's equation.