Product Distribution Field Theory

TL;DR

This paper introduces a new approximation method extending mean field theory to better capture dependencies in complex systems, leading to novel algorithms for energy minimization, optimization, and control.

Contribution

It develops a generalized product distribution approximation framework that captures dependencies and facilitates Monte Carlo estimation, advancing energy-minimization algorithms and game theory concepts.

Findings

New energy-minimization algorithm with coordinate Monte Carlo estimation

Application to high-dimensional integration and combinatorial optimization

Provides insights for improved algorithms and a bounded rationality equilibrium concept

Abstract

This paper presents a novel way to approximate a distribution governing a system of coupled particles with a product of independent distributions. The approach is an extension of mean field theory that allows the independent distributions to live in a different space from the system, and thereby capture statistical dependencies in that system. It also allows different Hamiltonians for each independent distribution, to facilitate Monte Carlo estimation of those distributions. The approach leads to a novel energy-minimization algorithm in which each coordinate Monte Carlo estimates an associated spectrum, and then independently sets its state by sampling a Boltzmann distribution across that spectrum. It can also be used for high-dimensional numerical integration, (constrained) combinatorial optimization, and adaptive distributed control. This approach also provides a simple, physics-based derivation of the powerful approximate energy-minimization algorithms semi-formally derived in \cite{wowh00, wotu02c, wolp03a}. In addition it suggests many improvements to those algorithms, and motivates a new (bounded rationality) game theory equilibrium concept.