Vector-wise Joint Diagonalization of Almost Commuting Matrices

TL;DR

This paper introduces a fast and robust vector-wise joint diagonalization algorithm for almost commuting matrices, with applications in independent component analysis, supported by theoretical analysis and numerical experiments.

Contribution

It proposes a novel VJD algorithm based on approximate common eigenvectors and a Riemannian quasi-Newton method, improving robustness and efficiency.

Findings

The VJD algorithm effectively diagonalizes almost commuting matrices.

Numerical experiments show favorable comparison with existing methods.

Application to independent component analysis demonstrates practical utility.

Abstract

This work aims to numerically construct exactly commuting matrices close to given almost commuting ones, which is equivalent to the joint approximate diagonalization problem. We first prove that almost commuting matrices generically have approximate common eigenvectors that are almost orthogonal to each other. Based on this key observation, we propose a fast and robust vector-wise joint diagonalization (VJD) algorithm, which constructs the orthogonal similarity transform by sequentially finding these approximate common eigenvectors. In doing so, we consider sub-optimization problems over the unit sphere, for which we present a Riemannian quasi-Newton method with rigorous convergence analysis. We also discuss the numerical stability of the proposed VJD algorithm. Numerical examples with applications in independent component analysis are provided to reveal the relation with Huaxin Lin's theorem and to demonstrate that our method compares favorably with the state-of-the-art Jacobi-type joint diagonalization algorithm.