Some observations on the ambivalent role of symmetries in Bayesian inference problems

TL;DR

This paper discusses how symmetries in Bayesian inference can be either helpful or harmful, emphasizing the importance of accounting for unobservable invariances to improve inference accuracy and analysis.

Contribution

It highlights the impact of symmetries on Bayesian inference and underscores the necessity to remove unobservable invariances in defining distances and analyzing models.

Findings

Symmetries can be beneficial or detrimental depending on their action.

Removing unobservable invariances is crucial for accurate inference.

Implications for statistical mechanics approaches in inference models.

Abstract

We collect in this note some observations on the role of symmetries in Bayesian inference problems, that can be useful or detrimental depending on the way they act on the signal and on the observations. We emphasize in particular the need to gauge away unobservable invariances in the definition of a distance between a signal and its estimator, and the consequences this implies for the statistical mechanics treatment of such models, taking as a motivating example the extensive rank matrix factorization problem.