On joint eigen-decomposition of matrices

TL;DR

This paper provides a theoretical analysis of the joint eigen-decomposition problem, proving the well-posedness of the optimization functional and deriving derivatives to improve numerical solutions, especially for self-adjoint matrices.

Contribution

It offers a rigorous proof of the cost-functional's behavior near rank-deficient matrices and derives explicit derivatives to enhance numerical algorithms.

Findings

The cost-functional tends to infinity near rank-deficient matrices with probability one.

Explicit formulas for gradients and Hessians are provided for the optimization problem.

The analysis includes specialized results for self-adjoint matrices.

Abstract

The problem of approximate joint diagonalization of a collection of matrices arises in a number of diverse engineering and signal processing problems. This problem is usually cast as an optimization problem, and it is the main goal of this publication to provide a theoretical study of the corresponding cost-functional. As our main result, we prove that this functional tends to infinity in the vicinity of rank-deficient matrices with probability one, thereby proving that the optimization problem is well posed. Secondly, we provide unified expressions for its higher-order derivatives in multilinear form, and explicit expressions for the gradient and the Hessian of the functional in standard form, thereby opening for new improved numerical schemes for the solution of the joint diagonalization problem. A special section is devoted to the important case of self-adjoint matrices.