Polynomial methods in statistical inference: theory and practice

TL;DR

This survey explores polynomial techniques in statistical inference, highlighting their theoretical foundations and practical applications in property estimation, mixture models, and establishing fundamental limits.

Contribution

It provides a comprehensive overview of polynomial methods, including approximation, interpolation, and orthogonal polynomials, and demonstrates their effectiveness in solving complex statistical problems.

Findings

Effective in entropy and support size estimation

Useful for learning Gaussian mixture models

Establishes fundamental limits of inference problems

Abstract

This survey provides an exposition of a suite of techniques based on the theory of polynomials, collectively referred to as polynomial methods, which have recently been applied to address several challenging problems in statistical inference successfully. Topics including polynomial approximation, polynomial interpolation and majorization, moment space and positive polynomials, orthogonal polynomials and Gaussian quadrature are discussed, with their major probabilistic and statistical applications in property estimation on large domains and learning mixture models. These techniques provide useful tools not only for the design of highly practical algorithms with provable optimality, but also for establishing the fundamental limits of the inference problems through the method of moment matching. The effectiveness of the polynomial method is demonstrated in concrete problems such as entropy and support size estimation, distinct elements problem, and learning Gaussian mixture models.