Towards algebraic methods for maximum entropy estimation

TL;DR

This paper demonstrates how maximum entropy estimation problems with integer-valued features can be transformed into polynomial systems solvable by Grobner bases, offering a new algebraic approach.

Contribution

It introduces an algebraic framework for maximum entropy estimation by linking it to solving polynomial equations using Grobner bases, applicable when features are integer-valued.

Findings

Maximum entropy estimation can be reformulated as solving polynomial systems.

Grobner bases provide a computational method for these polynomial systems.

Applicable to models with integer-valued feature functions.

Abstract

We show that various formulations (e.g., dual and Kullback-Csiszar iterations) of estimation of maximum entropy (ME) models can be transformed to solving systems of polynomial equations in several variables for which one can use celebrated Grobner bases methods. Posing of ME estimation as solving polynomial equations is possible, in the cases where feature functions (sufficient statistic) that provides the information about the underlying random variable in the form of expectations are integer valued.