Relations between redundancy patterns of the Shannon code and wave diffraction patterns of partially disordered media

TL;DR

This paper explores the connection between the redundancy patterns of Shannon coding and wave diffraction in disordered media, revealing how structural imperfections influence information redundancy similarly to diffraction patterns in physics.

Contribution

It introduces an analogy between Shannon code redundancy behavior and wave diffraction patterns, providing new insights into the effects of disorder on information encoding.

Findings

Redundancy exhibits oscillatory or convergent behavior depending on source parameters.

Perfect crystals produce sharp diffraction peaks analogous to oscillatory redundancy.

Imperfect structures lead to diffuse diffraction, paralleling convergent redundancy patterns.

Abstract

The average redundancy of the Shannon code, $R_n$, as a function of the block length $n$, is known to exhibit two very different types of behavior, depending on the rationality or irrationality of certain parameters of the source: It either converges to 1/2 as $n$ grows without bound, or it may have a non-vanishing, oscillatory, (quasi-) periodic pattern around the value 1/2 for all large $n$. In this paper, we make an attempt to shed some insight into this erratic behavior of $R_n$, by drawing an analogy with the realm of physics of wave propagation, in particular, the elementary theory of scattering and diffraction. It turns out that there are two types of behavior of wave diffraction patterns formed by crystals, which are correspondingly analogous to the two types of patterns of $R_n$. When the crystal is perfect, the diffraction intensity spectrum exhibits very sharp peaks, a.k.a. Bragg peaks, at wavelengths of full constructive interference. These wavelengths correspond to the frequencies of the harmonic waves of the oscillatory mode of $R_n$. On the other hand, when the crystal is imperfect and there is a considerable degree of disorder in its structure, the Bragg peaks disappear, and the behavior of this mode is analogous to the one where $R_n$ is convergent.