Semi-Empirical Objective Functions for MCMC Proposal Optimization

TL;DR

This paper introduces semi-empirical Ab Initio objective functions for neural MCMC proposal optimization, enabling the use of highly expressive proposal architectures without architectural restrictions, and demonstrating improved optimization and performance.

Contribution

The paper presents a novel semi-empirical approach to designing objective functions for neural MCMC proposals that are flexible and robust, surpassing existing methods.

Findings

Ab Initio objective functions outperform existing objectives in neural MCMC optimization.

They are robust and enable effective training of deep generative proposal networks.

Experimental results show improved MCMC efficiency and optimization stability.

Abstract

Current objective functions used for training neural MCMC proposal distributions implicitly rely on architectural restrictions to yield sensible optimization results, which hampers the development of highly expressive neural MCMC proposal architectures. In this work, we introduce and demonstrate a semi-empirical procedure for determining approximate objective functions suitable for optimizing arbitrarily parameterized proposal distributions in MCMC methods. Our proposed Ab Initio objective functions consist of the weighted combination of functions following constraints on their global optima and transformation invariances that we argue should be upheld by general measures of MCMC efficiency for use in proposal optimization. Our experimental results demonstrate that Ab Initio objective functions maintain favorable performance and preferable optimization behavior compared to existing objective functions for neural MCMC optimization. We find that Ab Initio objective functions are sufficiently robust to enable the confident optimization of neural proposal distributions parameterized by deep generative networks extending beyond the regimes of traditional MCMC schemes