Partial exchangeability of the prior via shuffling

TL;DR

This paper introduces a shuffling technique that leverages partial exchangeability in multi-dimensional parameter inference, leading to a novel shrinkage estimator that improves Bayesian risk properties without extra prior assumptions.

Contribution

The paper presents a symmetrization method called shuffling for incorporating partial exchangeability into models, and develops an EM algorithm for estimating parameter multisets, resulting in a new shrinkage estimator.

Findings

Shuffling induces a form of shrinkage estimator in exponential family models.

The method aligns Bayesian risk with invariance under permutation groups.

Empirical results demonstrate improved estimation accuracy.

Abstract

In inference problems involving a multi-dimensional parameter $\theta$, it is often natural to consider decision rules that have a risk which is invariant under some group $G$ of permutations of $\theta$. We show that this implies that the Bayes risk of the rule is {\em as if} the prior distribution of the parameter is partially exchangeable with respect to $G$. We provide a symmetrization technique for incorporating partial exchangeability of $\theta$ into a statistical model, without assuming any other prior information. We refer to this technique as {\em shuffling}. Shuffling can be viewed as an instance of empirical Bayes, where we estimate the (unordered) multiset of parameter values $\{\theta_1,\theta_2,\dots,\theta_p\}$ while using a uniform prior on $G$ for their ordering. Estimation of the multiset is a missing data problem which can be tackled with a stochastic EM algorithm. We show that in the special case of estimating the mean-value parameter in a regular exponential family model, shuffling leads to an estimator that is a weighted average of permuted versions of the usual maximum likelihood estimator. This is a novel form of shrinkage.