The Curious Problem of the Normal Inverse Mean: Robustness and Shrinkage

TL;DR

This paper explores robust Bayesian methods for astronomical distance estimation from parallax data, demonstrating that heavy-tailed priors improve reliability especially with large measurement errors, and analyzing the theoretical limits of prior influence.

Contribution

It introduces a class of heavy-tailed priors for robust distance estimation, analyzing their theoretical properties and demonstrating their effectiveness with Gaia data.

Findings

Heavy-tailed priors reduce bias and variance in distance estimates.

The 'curse of a single observation' limits prior influence on the posterior.

Reciprocal invariant priors like Half-Cauchy are particularly effective.

Abstract

In astronomical observations, the estimation of distances from parallaxes is a challenging task due to the inherent measurement errors and the non-linear relationship between the parallax and the distance. This study leverages ideas from robust Bayesian inference to tackle these challenges, investigating a broad class of prior densities for estimating distances with a reduced bias and variance. Through theoretical analysis, simulation experiments, and the application to data from the Gaia Data Release 1 (GDR1), we demonstrate that heavy-tailed priors provide more reliable distance estimates, particularly in the presence of large fractional parallax errors. Theoretical results highlight the "curse of a single observation," where the likelihood dominates the posterior, limiting the impact of the prior. Nevertheless, heavy-tailed priors can delay the explosion of posterior risk, offering a more robust framework for distance estimation. The findings suggest that reciprocal invariant priors, with polynomial decay in their tails, such as the Half-Cauchy and Product Half-Cauchy, are particularly well-suited for this task, providing a balance between bias reduction and variance control.