Calculating entropy at different scales among diverse communication systems

TL;DR

This study compares entropy measures across different communication systems at various scales, demonstrating that the Fundamental Scale method effectively quantifies complexity in texts, music, and code, revealing structural differences.

Contribution

It introduces the Fundamental Scale approach for entropy measurement, applicable across diverse communication systems, providing a unified way to assess their complexity.

Findings

Fundamental Scale method aligns with word-based entropy in texts.

The method quantifies complexity in non-verbal systems like music.

Results reveal structural differences among communication systems.

Abstract

We evaluated the impact of changing the observation scale over the entropy measures for text descriptions. MIDI coded Music, computer code and two human natural languages were studied at the scale of characters, words, and at the Fundamental Scale resulting from adjusting the symbols length used to interpret each text-description until it produced minimum entropy. The results show that the Fundamental Scale method is comparable with the use of words when measuring entropy levels in written texts. However, this method can also be used in communication systems lacking words such as music. Measuring symbolic entropy at the fundamental scale allows to calculate quantitatively, relative levels of complexity for different communication systems. The results open novel vision on differences among the structure of the communication systems studied.