Maximum Entropy in the framework of Algebraic Statistics: A First Step

TL;DR

This paper explores the integration of maximum entropy models within algebraic statistics, demonstrating that under certain conditions, these models form affine varieties and are related to toric models, bridging information theory and algebraic geometry.

Contribution

It introduces a novel algebraic geometric framework for maximum entropy models, showing their structure as affine and toric models under specific conditions.

Findings

Maximum entropy models are affine varieties in algebraic statistics.

ME-models are toric models when constraint functions are integer-valued.

The paper establishes a connection between information theory and algebraic geometry.

Abstract

Algebraic statistics is a recently evolving field, where one would treat statistical models as algebraic objects and thereby use tools from computational commutative algebra and algebraic geometry in the analysis and computation of statistical models. In this approach, calculation of parameters of statistical models amounts to solving set of polynomial equations in several variables, for which one can use celebrated Grobner bases theory. Owing to the important role of information theory in statistics, this paper as a first step, explores the possibility of describing maximum and minimum entropy (ME) models in the framework of algebraic statistics. We show that ME-models are toric models (a class of algebraic statistical models) when the constraint functions (that provide the information about the underlying random variable) are integer valued functions, and the set of statistical models that results from ME-methods are indeed an affine variety.