Bayesian Empirical Bayes: Simultaneous Inference from Probabilistic Symmetries

TL;DR

This paper introduces Bayesian Empirical Bayes (BEB), a flexible framework leveraging probabilistic symmetries to extend empirical Bayes methods to complex structured data like arrays, graphs, and spatial processes.

Contribution

The paper proposes a generalized BEB approach based on probabilistic symmetry, extending classical EB to complex data structures with scalable algorithms.

Findings

BEB outperforms existing methods in denoising arrays and spatial data.

Demonstrated effectiveness on gene-expression and air-quality datasets.

Unified framework for diverse probabilistic symmetries.

Abstract

Empirical Bayes (EB) improves the accuracy of simultaneous inference "by learning from the experience of others" (Efron, 2012). Classical EB theory focuses on latent variables that are iid draws from a fitted prior (Efron, 2019). Modern applications, however, feature complex structure, like arrays, spatial processes, or covariates. How can we apply EB ideas to these settings? We propose a generalized approach to empirical Bayes based on the notion of probabilistic symmetry. Our method pairs a simultaneous inference problem-with an unknown prior-to a symmetry assumption on the joint distribution of the latent variables. Each symmetry implies an ergodic decomposition, which we use to derive a corresponding empirical Bayes method. We call this methodBayesian empirical Bayes (BEB). We show how BEB recovers the classical methods of empirical Bayes, which implicitly assume exchangeability. We then use it to extend EB to other probabilistic symmetries: (i) EB matrix recovery for arrays and graphs; (ii) covariate-assisted EB for conditional data; (iii) EB spatial regression under shift invariance. We develop scalable algorithms based on variational inference and neural networks. In simulations, BEB outperforms existing approaches to denoising arrays and spatial data. On real data, we demonstrate BEB by denoising a cancer gene-expression matrix and analyzing spatial air-quality data from New York City.