Computational approaches for empirical Bayes methods and Bayesian sensitivity analysis

TL;DR

This paper develops computational methods combining importance sampling and control variates to efficiently perform sensitivity analysis and prior subset identification in Bayesian models, with applications to variable selection.

Contribution

It introduces a general methodology for simultaneous posterior expectation estimation across priors and prior subset selection using Markov chain samples.

Findings

Method effectively estimates posterior expectations for all priors in the family.

Applicable to Bayesian linear regression and variable selection models.

Demonstrated on US crime data example.

Abstract

We consider situations in Bayesian analysis where we have a family of priors $\nu_h$ on the parameter $\theta$, where $h$ varies continuously over a space $\mathcal{H}$, and we deal with two related problems. The first involves sensitivity analysis and is stated as follows. Suppose we fix a function $f$ of $\theta$. How do we efficiently estimate the posterior expectation of $f(\theta)$ simultaneously for all $h$ in $\mathcal{H}$? The second problem is how do we identify subsets of $\mathcal{H}$ which give rise to reasonable choices of $\nu_h$? We assume that we are able to generate Markov chain samples from the posterior for a finite number of the priors, and we develop a methodology, based on a combination of importance sampling and the use of control variates, for dealing with these two problems. The methodology applies very generally, and we show how it applies in particular to a commonly used model for variable selection in Bayesian linear regression, and give an illustration on the US crime data of Vandaele.