A Bayesian Approach to Approximate Joint Diagonalization of Square Matrices

TL;DR

This paper introduces a Bayesian Gibbs sampling method for approximate joint diagonalization of multiple square matrices, demonstrating superior performance and applications in source separation and PCA.

Contribution

The paper develops a novel Bayesian Gibbs sampler for joint diagonalization, outperforming existing algorithms and enabling applications in signal processing tasks.

Findings

Achieves state-of-the-art performance in synthetic examples

Accurately estimates the number of common eigenvectors using BIC

Successfully applied to source separation and PCA problems

Abstract

We present a Bayesian scheme for the approximate diagonalisation of several square matrices which are not necessarily symmetric. A Gibbs sampler is derived to simulate samples of the common eigenvectors and the eigenvalues for these matrices. Several synthetic examples are used to illustrate the performance of the proposed Gibbs sampler and we then provide comparisons to several other joint diagonalization algorithms, which shows that the Gibbs sampler achieves the state-of-the-art performance on the examples considered. As a byproduct, the output of the Gibbs sampler could be used to estimate the log marginal likelihood, however we employ the approximation based on the Bayesian information criterion (BIC) which in the synthetic examples considered correctly located the number of common eigenvectors. We then succesfully applied the sampler to the source separation problem as well as the common principal component analysis and the common spatial pattern analysis problems.