Improving Grid Based Bayesian Methods

TL;DR

This paper introduces a novel approach to hyperparameter exploration using low discrepancy point sets, polynomial marginal estimation, and functional decomposition to improve efficiency and accuracy in Bayesian methods.

Contribution

It presents a new deterministic grid exploration method with low discrepancy points, a polynomial-based marginal estimation technique, and the use of anchored f-ANOVA for enhanced accuracy.

Findings

Low discrepancy points enable accurate marginal estimation at lower computational cost.

Polynomial least squares methods converge to true marginals under certain conditions.

Functional decomposition improves the efficiency and accuracy of hyperparameter integration.

Abstract

In some cases, computational benefit can be gained by exploring the hyper parameter space using a deterministic set of grid points instead of a Markov chain. We view this as a numerical integration problem and make three unique contributions. First, we explore the space using low discrepancy point sets instead of a grid. This allows for accurate estimation of marginals of any shape at a much lower computational cost than a grid based approach and thus makes it possible to extend the computational benefit to a hyper parameter space with higher dimensionality (10 or more). Second, we propose a new, quick and easy method to estimate the marginal using a least squares polynomial and prove the conditions under which this polynomial will converge to the true marginal. Our results are valid for a wide range of point sets including grids, random points and low discrepancy points. Third, we show that further accuracy and efficiency can be gained by taking into consideration the functional decomposition of the integrand and illustrate how this can be done using anchored f-ANOVA on weighted spaces.