On Minimax Optimality of Sparse Bayes Predictive Density Estimates

TL;DR

This paper investigates the asymptotic minimaxity of Bayesian predictive density estimates in sparse Gaussian models, revealing a phase transition related to the variance ratio and proposing priors that achieve optimality.

Contribution

It introduces a phase transition phenomenon in predictive density estimation and proposes new priors that attain asymptotic minimaxity in sparse models.

Findings

Existence of a phase transition at a critical variance ratio r_0.

Bi-grid prior with reduced grid spacing achieves minimaxity for r < r_0.

Spike-and-slab priors can be minimax under magnitude constraints.

Abstract

We study predictive density estimation under Kullback-Leibler loss in $\ell_0$-sparse Gaussian sequence models. We propose proper Bayes predictive density estimates and establish asymptotic minimaxity in sparse models. A surprise is the existence of a phase transition in the future-to-past variance ratio $r$. For $r < r_0 = (\surd 5 - 1)/4$, the natural discrete prior ceases to be asymptotically optimal. Instead, for subcritical $r$, a `bi-grid' prior with a central region of reduced grid spacing recovers asymptotic minimaxity. This phenomenon seems to have no analog in the otherwise parallel theory of point estimation of a multivariate normal mean under quadratic loss. For spike-and-slab priors to have any prospect of minimaxity, we show that the sparse parameter space needs also to be magnitude constrained. Within a substantial range of magnitudes, spike-and-slab priors can attain asymptotic minimaxity.