The geometry of Stein's method of moments: A canonical decomposition via score matching

TL;DR

This paper explores the geometric structure of Stein's method of moments, revealing a canonical decomposition that enhances estimator efficiency and connects to Wasserstein geometry, advancing understanding of parameter estimation without normalizing constants.

Contribution

It introduces a canonical decomposition of SMoM estimators, linking score matching to Wasserstein geometry and improving asymptotic variance performance.

Findings

Decomposition clarifies the role of score matching in SMoM

Constructed an estimator with lower asymptotic variance

Connected SMoM to Wasserstein geometry and efficiency conditions

Abstract

In this paper, we elucidate the geometry of Stein's method of moments (SMoM). SMoM is a parameter estimation method based on the Stein operator, and yields a wide class of estimators that do not depend on the normalizing constant. We present a canonical decomposition of an SMoM estimator after centering the score matching estimator, which sheds light on the central role of the score matching within the SMoM framework. Using this decomposition, we construct an SMoM estimator that improves upon the score matching estimator in the asymptotic variance. We also discuss the connection between SMoM and the Wasserstein geometry. Specifically, using the Wasserstein score function, we provide a geometrical interpretation of the gap in the asymptotic variance between the score matching estimator and the maximum likelihood estimator. Furthermore, it is shown that the score matching estimator is asymptotically efficient if and only if the Fisher score functions span the same space as the Wasserstein score functions.