Newton's Algorithm as a Gradient Flow: A Geometric Framework for Recursive Mixture Estimation

TL;DR

This paper offers a geometric interpretation of Newton's recursive mixture estimation algorithm, connecting it to gradient flows on probability spaces, and provides a theoretical foundation for its convergence and potential extensions.

Contribution

It reveals that Newton's recursion approximates a gradient flow on probability measures, establishing a rigorous dynamical framework and enabling systematic analysis and generalization.

Findings

Newton's recursion is a discrete approximation of a Fisher-Rao gradient flow.

The geometric perspective clarifies convergence properties of the estimator.

This framework links recursive Bayesian estimators to variational Bayes methods.

Abstract

Bayesian nonparametric mixture models provide a flexible framework for data analysis but are often hindered by the computational expense of traditional inference methods like MCMC. A fast, recursive algorithm proposed by Newton (2002) offers a practical alternative, yet its formal connection to Bayesian inference and its theoretical properties remain only partially understood. This paper reveals a new geometric interpretation of this class of predictive recursions. We demonstrate that Newton's recursion is a discrete-time approximation of a gradient flow on the space of probability measures governed by the Fisher-Rao geometry, providing the first rigorous dynamical characterisation of this family of estimators. This geometric perspective provides a principled theoretical foundation for studying these recursions: it clarifies their convergence behaviour, situates them within the variational Bayes literature, and yields a systematic basis for generalisation by modifying the underlying geometry and discretisation. In contrast to approaches that construct gradient flows from a prescribed variational objective, this work proceeds in the reverse direction: beginning from an existing recursive estimator and uncovering the variational problem it implicitly solves, it opens a pathway for the systematic analysis and extension of a broad class of sequential Bayesian estimators.