Estimating a potential without the agony of the partition function

TL;DR

This paper introduces the MR-MAP approach, an efficient method for estimating Gibbs densities without computing the intractable partition function, using neural networks for fast optimization.

Contribution

It proposes the MR-MAP estimator that avoids partition function calculation and reformulates density estimation as a neural network-based optimization.

Findings

MR-MAP effectively estimates densities without partition function computation

The neural network approach enables fast and scalable optimization

Experimental results demonstrate competitive accuracy on standard datasets

Abstract

Estimating a Gibbs density function given a sample is an important problem in computational statistics and statistical learning. Although the well established maximum likelihood method is commonly used, it requires the computation of the partition function (i.e., the normalization of the density). This function can be easily calculated for simple low-dimensional problems but its computation is difficult or even intractable for general densities and high-dimensional problems. In this paper we propose an alternative approach based on Maximum A-Posteriori (MAP) estimators, we name Maximum Recovery MAP (MR-MAP), to derive estimators that do not require the computation of the partition function, and reformulate the problem as an optimization problem. We further propose a least-action type potential that allows us to quickly solve the optimization problem as a feed-forward hyperbolic neural network. We demonstrate the effectiveness of our methods on some standard data sets.