Bayesimax Theory: Selecting Priors by Minimizing Total Information

TL;DR

Bayesimax theory introduces a method for selecting noninformative priors by applying minimax principles to prior disclosure games, aiming to minimize total information and maximize conditional entropy.

Contribution

The paper develops a novel objective Bayesian framework using minimax theory and proper scoring rules to derive priors that minimize total information and maximize entropy.

Findings

Bayesimax priors are shown to be noninformative and optimal under certain conditions.

The theory connects minimax analysis with entropy maximization, providing a new perspective on prior selection.

A Monte Carlo algorithm is proposed for estimating conditional entropy under these priors.

Abstract

We introduce Bayesimax theory, a paradigm for objective Bayesian analysis which selects priors by applying minimax theory to prior disclosure games. In these games, the uniquely optimal strategy for a Bayesian agent upon observing the data is to reveal their prior. As such, the prior chosen by minimax theory is, in effect, the implicit prior of minimax agents. Since minimax analysis disregards prior information, this prior is arguably noninformative. We refer to minimax solutions of certain prior disclosure games as Bayesimax priors, and we classify a statistical procedure as Bayesimax if it is a Bayes rule with respect to a Bayesimax prior. Under regular conditions, if a decision rule is minimax, then it is a Bayes rule under priors which maximize the minimum Bayes risk. We study games leveraging strictly proper scoring rules to induce posterior (and thereby prior) revelation. In such games, the minimum Bayes risk equals the conditional (generalized) entropy of the parameter given the data. Bayesimax theory therefore prescribes conditional entropy maximization. As conditional entropy equals marginal entropy (prior uninformativeness) minus mutual information (data informativeness), Bayesimax priors effectively minimize total information. We provide a rigorous formulation of these ideas, characterize sufficient conditions for regularity and identifiability, and investigate asymptotics and conjugate family examples. We next describe a generic Monte Carlo algorithm for estimating conditional entropy under a given prior. Finally, we compare and contrast Bayesimax theory with various related proposals from the objective Bayes and robust Bayes literature.