On the Geometry of Maximum Entropy Problems

TL;DR

This paper reveals a geometric principle that simplifies deriving solutions for a broad class of maximum entropy problems involving probability distributions, spectral densities, and covariance matrices, including new covariance completion scenarios.

Contribution

It introduces a geometric orthogonality principle that unifies and extends existing maximum entropy solution methods and applies it to novel covariance matrix completion problems.

Findings

Derived a general form for maximum entropy solutions using geometry.

Unified existing methods like Burg's spectral estimation and Dempster's covariance completion.

Extended the approach to block-circulant covariance matrix completion.

Abstract

We show that a simple geometric result suffices to derive the form of the optimal solution in a large class of finite and infinite-dimensional maximum entropy problems concerning probability distributions, spectral densities and covariance matrices. These include Burg's spectral estimation method and Dempster's covariance completion, as well as various recent generalizations of the above. We then apply this orthogonality principle to the new problem of completing a block-circulant covariance matrix when an a priori estimate is available.