Entropy, Perception, and Relativity

TL;DR

This paper introduces a generalized entropy concept integrating information from multiple events, links perception to true uncertainty via a yin-yang model, and applies these ideas to neural networks and relativity.

Contribution

It proposes a new form of entropy extending Shannon's definition, models perception as a yin-yang balance, and connects this to neural network functions and Einstein's time dilation.

Findings

Perceived uncertainty aligns with true uncertainty at points defined by the golden ratio.

Sigmoid functions in neural networks reflect actual performance according to the new entropy model.

Theoretical framework applied to pattern recognition and network architecture design.

Abstract

In this paper, I expand Shannon's definition of entropy into a new form of entropy that allows integration of information from different random events. Shannon's notion of entropy is a special case of my more general definition of entropy. I define probability using a so-called performance function, which is de facto an exponential distribution. Assuming that my general notion of entropy reflects the true uncertainty about a probabilistic event, I understand that our perceived uncertainty differs. I claim that our perception is the result of two opposing forces similar to the two famous antagonists in Chinese philosophy: Yin and Yang. Based on this idea, I show that our perceived uncertainty matches the true uncertainty in points determined by the golden ratio. I demonstrate that the well-known sigmoid function, which we typically employ in artificial neural networks as a non-linear threshold function, describes the actual performance. Furthermore, I provide a motivation for the time dilation in Einstein's Special Relativity, basically claiming that although time dilation conforms with our perception, it does not correspond to reality. At the end of the paper, I show how to apply this theoretical framework to practical applications. I present recognition rates for a pattern recognition problem, and also propose a network architecture that can take advantage of general entropy to solve complex decision problems.