The Relation Between the EVD or SVD of Summands and the EVD or SVD of the Sum

TL;DR

This paper explores how the eigenstructures of summands relate to the eigenstructure of their sum, providing methods to derive eigen and singular value decompositions of sums and inner products from those of individual matrices.

Contribution

It introduces a novel approach to express the eigen and singular value decompositions of a sum of matrices using the decompositions of the summands, facilitating computations in complex eigendecomposition problems.

Findings

Eigenstructure of sum can be derived from block matrix projections.

SVD of inner product can be expressed via eigenstructures of factors.

Method generalizes to sums of arbitrary matrices for eigendecomposition.

Abstract

In this work, we show how the eigenstructures of summands are related to that of the sum. In particular, we show that the sum of two positive semidefinite matrices can be written as the inner product of two block matrices $\mathbf{C} = \mathbf{A} + \mathbf{B} = \mathbf{PD}^2\mathbf{P}^T + \mathbf{QE}^2\mathbf{Q}^T = \begin{pmatrix} \mathbf{PD} & \mathbf{QE} \end{pmatrix}\begin{pmatrix} \mathbf{PD} & \mathbf{QE} \end{pmatrix}^T = \mathbf{Z}^T\mathbf{Z}$, such that the eigenvector decomposition of $\mathbf{C}$ can be obtained by projecting the eigenvectors from the block matrix product $\mathbf{ZZ}^T$ onto the block matrix $\mathbf{Z}^T = \begin{pmatrix} \mathbf{PD} & \mathbf{QE} \end{pmatrix}$. Next, it is shown that the result can be used to rewrite the SVD of a matrix inner product $\mathbf{X}^T\mathbf{Y}$, utilizing the eigenstructures of $\mathbf{X}$ and $\mathbf{Y}$. Finally, it is shown how this can be generalized to express the SVD of a sum of arbitrary matrices $\mathbf{H} = \mathbf{F} + \mathbf{G}$ in terms of the SVDs of the summands. The results may be useful in algorithms for eigendecomposition, specifically if the eigenproblem can be expressed in terms of a sum of matrices with simple eigenstructures as, e.g., in multi-omics research and generalized Procrustes analysis.