Incomplete Reparameterizations and Equivalent Metrics

TL;DR

This paper investigates how incomplete reparameterizations in probabilistic systems affect algorithm performance, revealing they alter interactions and can be viewed as modifications to the underlying metric geometry, especially in MCMC on Riemannian manifolds.

Contribution

It demonstrates that common incomplete reparameterizations change system interactions and are equivalent to metric modifications in Riemannian manifold MCMC.

Findings

Incomplete reparameterizations alter algorithm interactions.

Reparameterizations can be viewed as metric modifications.

Implications for Riemannian manifold MCMC algorithms.

Abstract

Reparameterizing a probabilisitic system is common advice for improving the performance of a statistical algorithm like Markov chain Monte Carlo, even though in theory such reparameterizations should leave the system, and the performance of any algorithm, invariant. In this paper I show how the reparameterizations common in practice are only incomplete reparameterizations which result in different interactions between a target probabilistic system and a given algorithm. I then consider how these changing interactions manifest in the context of Markov chain Monte Carlo algorithms defined on Riemannian manifolds. In particular I show how any incomplete reparameterization is equivalent to modifying the metric geometry directly.