Uncertain Reasoning Using Maximum Entropy Inference

TL;DR

This paper examines the justification of maximum entropy inference in uncertain reasoning, addresses objections, compares it with other methods, and advocates for a combined approach with probability theory for a more grounded theory.

Contribution

It clarifies the role of maximum entropy as a dynamic reasoning method and proposes integrating it with static probability theory for a comprehensive uncertain reasoning framework.

Findings

Maximum entropy is a dynamic theory of belief updating.

Combining maximum entropy with probability theory yields a more grounded reasoning framework.

Addresses and responds to objections to the information-theoretic justification.

Abstract

The use of maximum entropy inference in reasoning with uncertain information is commonly justified by an information-theoretic argument. This paper discusses a possible objection to this information-theoretic justification and shows how it can be met. I then compare maximum entropy inference with certain other currently popular methods for uncertain reasoning. In making such a comparison, one must distinguish between static and dynamic theories of degrees of belief: a static theory concerns the consistency conditions for degrees of belief at a given time; whereas a dynamic theory concerns how one's degrees of belief should change in the light of new information. It is argued that maximum entropy is a dynamic theory and that a complete theory of uncertain reasoning can be gotten by combining maximum entropy inference with probability theory, which is a static theory. This total theory, I argue, is much better grounded than are other theories of uncertain reasoning.