MCMC Algorithms for Posteriors on Matrix Spaces

TL;DR

This paper develops a theoretical framework for MCMC algorithms on matrix spaces, introduces a new algorithm called MpCN, and demonstrates its robustness and empirical performance on heavy-tailed distributions, with applications in financial data modeling.

Contribution

The paper introduces the MpCN algorithm for matrix space distributions and analyzes its ergodicity properties, filling a gap in the theoretical understanding of MCMC on matrix spaces.

Findings

RWM and pCN are not geometrically ergodic for heavy-tailed matrix distributions.

MpCN is robust and performs well empirically across various heavy-tailed targets.

Progress is made towards proving geometric ergodicity of MpCN, with some steps remaining for future work.

Abstract

We study Markov chain Monte Carlo (MCMC) algorithms for target distributions defined on matrix spaces. Such an important sampling problem has yet to be analytically explored. We carry out a major step in covering this gap by developing the proper theoretical framework that allows for the identification of ergodicity properties of typical MCMC algorithms, relevant in such a context. Beyond the standard Random-Walk Metropolis (RWM) and preconditioned Crank--Nicolson (pCN), a contribution of this paper in the development of a novel algorithm, termed the `Mixed' pCN (MpCN). RWM and pCN are shown not to be geometrically ergodic for an important class of matrix distributions with heavy tails. In contrast, MpCN is robust across targets with different tail behaviour and has very good empirical performance within the class of heavy-tailed distributions. Geometric ergodicity for MpCN is not fully proven in this work, as some remaining drift conditions are quite challenging to obtain owing to the complexity of the state space. We do, however, make a lot of progress towards a proof, and show in detail the last steps left for future work. We illustrate the computational performance of the various algorithms through numerical applications, including calibration on real data of a challenging model arising in financial statistics.