Entropic Inference: some pitfalls and paradoxes we can avoid

TL;DR

This paper examines the pitfalls and paradoxes in entropic inference, emphasizing the importance of correctly understanding constraints to avoid errors and improve the method's reliability.

Contribution

It introduces a classification of four types of constraints in entropic inference and discusses how misinterpretations lead to common paradoxes and failures.

Findings

Misunderstanding constraint types causes errors in inference.

Replacing expected values with sample averages can be problematic.

Analyzing the Friedman-Shimony paradox clarifies entropic inference issues.

Abstract

The method of maximum entropy has been very successful but there are cases where it has either failed or led to paradoxes that have cast doubt on its general legitimacy. My more optimistic assessment is that such failures and paradoxes provide us with valuable learning opportunities to sharpen our skills in the proper way to deploy entropic methods. The central theme of this paper revolves around the different ways in which constraints are used to capture the information that is relevant to a problem. This leads us to focus on four epistemically different types of constraints. I propose that the failure to recognize the distinctions between them is a prime source of errors. I explicitly discuss two examples. One concerns the dangers involved in replacing expected values with sample averages. The other revolves around misunderstanding ignorance. I discuss the Friedman-Shimony paradox as it is manifested in the three-sided die problem and also in its original thermodynamic formulation.