Power-Laws and the Conservation of Information in discrete token systems: Part 1 General Theory

TL;DR

This paper proposes that the Conservation of Hartley-Shannon Information underpins power-law distributions in discrete systems like software and biological genomes, analogous to energy conservation in physics.

Contribution

It introduces a new conservation principle for information in discrete systems and demonstrates its implications for power-law behavior and biological constants.

Findings

Power-law distributions in component sizes are universal in software systems.

Average gene length remains constant across biological systems.

Information conservation explains observed patterns in diverse discrete systems.

Abstract

The Conservation of Energy plays a pivotal part in the development of the physical sciences. With the growth of computation and the study of other discrete token based systems such as the genome, it is useful to ask if there are conservation principles which apply to such systems and what kind of functional behaviour they imply for such systems. Here I propose that the Conservation of Hartley-Shannon Information plays the same over-arching role in discrete token based systems as the Conservation of Energy does in physical systems. I will go on to prove that this implies power-law behaviour in component sizes in software systems no matter what they do or how they were built, and also implies the constancy of average gene length in biological systems as reported for example by Lin Xu et al (10.1093/molbev/msk019). These propositions are supported by very large amounts of experimental data extending the first presentation of these ideas in Hatton (2011, IFIP / SIAM / NIST Working Conference on Uncertainty Quantification in Scientific Computing, Boulder, August 2011).