A historical view on the maximum entropy

TL;DR

This paper reviews the maximum entropy principle, discussing its features, advantages, disadvantages, and extensions like generalized maximum entropy, to understand its role in estimating unknown distributions.

Contribution

It provides a comprehensive review of the maximum entropy principle, including its features, extensions, and limitations, offering a balanced perspective on its applicability.

Findings

Maximum entropy effectively estimates unknown distributions.

Generalized maximum entropy handles autocorrelation data.

The paper discusses both benefits and drawbacks of entropy methods.

Abstract

How to find unknown distributions is questioned in many pieces of research. There are several ways to figure them out, but the main question is which acts more reasonably than others. In this paper, we focus on the maximum entropy principle as a suitable method of discovering the unknown distribution, which recommends some prior information based on the available data set. We explain its features by reviewing some papers. Furthermore, we recommend some articles to study around the generalized maximum entropy issue, which is more suitable when autocorrelation data exists. Then, we list the beneficial features of the maximum entropy as a result. Finally, some disadvantages of entropy are expressed to have a complete look at the maximum entropy principle, and we list its drawbacks as the final step.