Solving the problem of simultaneous diagonalization of complex symmetric matrices via congruence

TL;DR

This paper presents a finite-step procedure to determine whether a set of complex symmetric matrices can be simultaneously diagonalized via congruence, solving a longstanding problem in the complex case.

Contribution

It reduces the simultaneous diagonalization via congruence problem to a related similarity problem, providing a practical finite-step solution.

Findings

Provides a finite-step procedure for diagonalization via congruence.

Reduces the problem to a classical similarity diagonalization problem.

Solves a long-standing open problem in the complex case.

Abstract

We provide a solution to the problem of simultaneous $diagonalization$ $via$ $congruence$ of a given set of $m$ complex symmetric $n\times n$ matrices $\{A_{1},\ldots,A_{m}\}$, by showing that it can be reduced to a possibly lower-dimensional problem where the question is rephrased in terms of the classical problem of simultaneous $diagonalization$ $via$ $similarity$ of a new related set of matrices. We provide a procedure to determine in a finite number of steps whether or not a set of matrices is simultaneously diagonalizable by congruence. This solves a long standing problem in the complex case.

Equations

$$P^T{A}_{i}Pis\; diagonal,\; for\; all1\le i\le m.$$

$$e_i{e}_j=\sum _{k=1}^n{m}_{ijk}{e}_k,i,j=1,\dots ,n,$$

$${\psi }_x(\tau ):=\ln E[\exp ({\tau }^T\overline{x})],\tau \in {\mathbb{C}}^n,$$

$${\psi }_x(\tau )=\ln E[\exp ((Q\tau {)}^T\overline{s})]=:{\psi }_s(\mu ),\mu =Q\tau .$$

$$(A_j{)}_{k\ell }:={\frac{{\partial }^2{\psi }_x(\tau )}{\partial {\tau }_k\partial {\tau }_{\ell }}|}_{\tau ={\tau }^{(j)}},k,\ell =1,\dots ,n,j=1,\dots ,m.$$

$$A_j=Q^T{D}_{j}Q,(D_j{)}_{k\ell }:={\frac{{\partial }^2{\psi }_s(\mu )}{\partial {\mu }_k\partial {\mu }_{\ell }}|}_{\mu =P{\tau }^{(j)}},k,\ell =1,\dots ,n,j=1,\dots ,m,$$

$$P^T{A}_{j}P=D_j,j=1,\dots ,m.$$

$$P^{-1}{L}_{j}P=D_j,j=1,\dots ,m.$$

$$A:{\mathbb{C}}^m⟶{\mathcal{M}}_{n}byA(\lambda )=\sum _{j=1}^m{\lambda }_j{A}_j,where\lambda =\left(\begin{array}{c}{\lambda }_1\\ ⋮\\ {\lambda }_m\end{array}\right)\in {\mathbb{C}}^m.$$

$$\underset{\begin{array}{c}\lambda \in {\mathbb{C}}^m\end{array}}{\sup }\mathrm{rank}A(\lambda )=\underset{\begin{array}{c}\lambda \in S^{2m-1}\end{array}}{\sup }\mathrm{rank}A(\lambda ).$$

$$\underset{\begin{array}{c}\lambda \in {\mathbb{C}}^m\end{array}}{\sup }\mathrm{rank}A(\lambda )=\underset{\begin{array}{c}\lambda \in S^{2m-1}\end{array}}{\sup }\mathrm{rank}A(\lambda )=\mathrm{rank}A({\lambda }_0).$$

$$\dim\big{(}\bigcap_{j=1}^{m}\mathrm{ker}A_{j}\big{)}=n-r\ \text{ if, and only if,}\ \bigcap_{j=1}^{m}\mathrm{ker}A_{j}=\mathrm{ker}{A}(\lambda_{0}).$$

$$\dim\big{(}\bigcap_{j=1}^{m}\mathrm{ker}A_{j}\big{)}\leq n-\mathrm{rank}A(\lambda),\ \text{for all}\ \lambda\in\mathbb{C}^{m}.$$

$$(P^{T}A(\lambda )P)(P^{-1}A(\lambda {)}^{-1}{A}_{j}P)=P^T{A}_{j}P,forj=1,\dots ,m,and\; anyP\in {\mathcal{G}\mathcal{L}}_n$$

$$A_1,\dots ,A_{m}are\; SDC\; if,\; and\; only\; if,A({\lambda }_0{)}^{-1}{A}_1,\dots ,A({\lambda }_0{)}^{-1}{A}_{m}are\; SDS.$$

$$P^{-1}A(\lambda_{0})^{-1}A_{j}P=(P^{T}A(\lambda_{0})P)^{-1}\big{(}P^{T}A_{j}P\big{)}$$

$$B_j=B({\lambda }_0)D^{(j)},j=1,\dots ,m.$$

$$\big{(}B(\lambda_{0})\big{)}_{k\ell}\big{(}D^{(j)}\big{)}_{\ell\ell}=\big{(}D^{(j)}\big{)}_{kk}\big{(}B(\lambda_{0})\big{)}_{k\ell},\ \text{for all}\ j=1,\ldots,m.$$

$$\big{(}B(\lambda_{0})\big{)}_{k\ell}=0\quad\text{if}\ \ \big{(}D^{(j)}\big{)}_{kk}\neq\big{(}D^{(j)}\big{)}_{\ell\ell}\ \text{for any}\ j=1,\ldots,m.$$

$${\alpha }_1^j=\dots ={\alpha }_{p_j}^j\ne {\alpha }_{p_j+1}^j$$

$${\alpha }_1^{(j)}={\alpha }_1^j=\dots ={\alpha }_{n_1}^j,for\; all1\le j\le m,$$

$$D^{(j)}={\alpha }_1^{(j)}{I}_{n_1}\oplus \mathrm{diag}({\alpha }_2^{(j)},{\alpha }_{n_1+2}^j,\dots ,{\alpha }_n^j),for\; all1\le j\le m,$$

$$D^{(j)}={\alpha }_1^{(j)}{I}_{n_1}\oplus {\alpha }_2^{(j)}{I}_{n_2}\oplus {\alpha }_3^{(j)}{I}_{n_3}\oplus \mathrm{diag}({\alpha }_{n_1+n_2+n_3+1}^j,\dots ,{\alpha }_n^j),$$

$$R^T{D}^{(j)}R=R^{-1}{D}^{(j)}R={\alpha }_1^{(j)}{I}_{n_1+n_3}\oplus {\alpha }_2^{(j)}{I}_{n_2}\oplus \mathrm{diag}({\alpha }_{n_1+n_2+n_3+1}^j,\dots ,{\alpha }_n^j),$$

$$U^T{D}^{(j)}U={\alpha }_1^{(j)}{I}_{p_1}\oplus \dots \oplus {\alpha }_d^{(j)}{I}_{p_d}$$

$$[U^T{D}^{(j)}U{]}_{aa}={\alpha }_a^{(j)}{I}_{n_a}$$

$$[U^{T}B({\lambda }_0)U{]}_{ab}=0\text{if}a\ne b.$$

$$U^{T}B({\lambda }_0)U=C_1\oplus \dots \oplus C_d,$$

$$V_a^T{C}_a{V}_a=D_a,1\le a\le d.$$

$$V^{T}\big{(}U^{T}B(\lambda_{0})U\big{)}V=D.$$

Full text

**This is a preprint of our work published in:

SIAM J. Matrix Anal. Appl., 41(4), 1616–1629 (October 2020).

https://doi.org/10.1137/19M1280430**

This work was partially supported by the Spanish Project MTM2016-76327-C3-2-P (AEI/FEDER,UE)

This work was also supported by the award of the Distinguished Visitor Grant of the School of Mathematics and Statistics, University College Dublin to the third author

1. Introduction

The aim of this paper is to characterise when a given set of $n\times n$ complex symmetric matrices, $A_{1},\ldots,A_{m}$, are simultaneously diagonalizable via congruence (SDC), namely, when there exists a non-singular $n\times n$ complex matrix $P$ such that

[TABLE]

Since Weierstrass in 1868 [24] gave sufficient conditions for the simultaneous diagonalization by congruence of two real symmetric matrices, several authors [7, 9, 14] have extended those results and there has been applications of the real results to areas as diverse as quadratic programming [13, 27, 2], variational analysis [8], signal processing [15, 16, 20] and medical imaging analysis [1, 5, 17, 23], among others. In the case of complex matrices, Hong, Horn and Johnson laid the framework in the 1980s for the particular case of unitary transformations by proving [10, 11] that there is a unitary $U$ satisfying $U^{T}A_{i}U$ is diagonal for all $i$ if, and only if, the set $\{A_{i}\overline{A_{j}}:1\leq i,j\leq m\}$ is a commuting family. In a more general case they solved the problem for pairs of complex symmetric (or Hermitian) matrices with the restriction that at least one of them be non-singular. We provide a solution in the general case of complex symmetric matrices by translating it into a simpler problem, at a possibly reduced dimension, regarding simultaneous diagonalizability by similarity of a new set of related matrices. We do this using the concept of matrix pencils so that the general problem is reduced to (possibly) lower dimensions a priori by calculating the intersection of the kernels of the matrices $A_{1},\ldots,A_{m}$. Once this is done, reduced $r\times r$ matrices ($r\leq n$) $\widetilde{A}_{i}$ (Lemma 10 below) can be dealt with in a more standard manner, thanks to the existence of a non-singular matrix pencil. This allows us to obtain fairly simple necessary and sufficient conditions for SDC in Theorem 14 below.

The authors were initially motivated to tackle SDC for complex symmetric matrices by a problem that arose naturally in the area of evolution algebras.

We recall here that an evolution algebra is defined as a commutative algebra $A$ over $\mathbb{C}$ for which there exists a basis $\widetilde{B}=\{\widetilde{e}_{i}:i\in\Lambda\}$ such that $\widetilde{e}_{i}\widetilde{e}_{j}=0$, for every $i,j\in\Lambda$ with $i\neq j$. In other words, the multiplication table of $A$ relative to $\widetilde{B}$ is diagonal. Such a basis is called natural. Evolution algebras were introduced in [19] and [18] in the study of non-Mendelian genetics and are, in general, not associative. The problem concerning the authors was to determine when a given algebra $A$ is an evolution algebra. In other words, if $B$ is a basis of $A$ and the multiplication table of $A$ with respect to $B$ is not diagonal, we established the conditions under which there exists a natural basis $\widetilde{B}$ of $A$, giving $A$ the structure of an evolution algebra. If $B=\{e_{1},...,e_{n}\}$ and

[TABLE]

we define the structure matrices of $A$ with respect to $B$ as the $n\times n$ matrices $M_{k}(B)=(m_{ijk})_{1\leq i,j\leq n},$ for $k=1,\ldots,n.$ Notice that the structure matrices $M_{k}(B)$ are symmetric and complex because $A$ is commutative and its base field is $\mathbb{C}$. A main result in [4] proves that $A$ is an evolution algebra if, and only if, the complex symmetric matrices $M_{1}(B),\ldots,M_{n}(B)$ are simultaneously diagonalizable via congruence.

There are other areas where the complex results might be applied. One of the most important applications is in the area of signal processing, in particular in the classical problem of blind source separation [3, 25, 26]. In its simplest form, and appropriate to our notation, this problem amounts to finding a nonsingular complex matrix $Q$ relating $n$ sets of measurements (denoted by the complex random vector $x$ of dimension $n$) and $n$ statistically independent, but unknown, sources (denoted by the complex random vector $s$ of dimension $n$), via the linear relation $x=Q^{*}s$, where $Q^{*}$ denotes the conjugate transpose of $Q$. To investigate how this relates to the SDC problem we discuss a method introduced in [25]. Consider a generalised second characteristic function, defined in terms of the $x$ variables as

[TABLE]

where bar denotes complex conjugation and $E$ denotes the expectation. In terms of the $s$ variables, this reads

[TABLE]

Let us consider $m$ so-called “processing points” $\tau^{(1)},\ldots,\tau^{(m)}$. We now define the following complex symmetric matrices $A_{1},\ldots,A_{m}$ by their components:

[TABLE]

It is then easy to show

[TABLE]

where the matrices $D_{j}$ are diagonal, due to the statistical independence of the components of $s$. This is, of course, the SDC problem, and its solution provides the complex matrix $Q$ that allows one to unveil the unknown independent sources starting from an arbitrary set of measurements. In real-life applications, experimental or numerical errors will lead to matrices $A_{1},\ldots,A_{m}$ that are not exactly SDC, so “approximate joint diagonalization” is the correct concept, which consists of the variational problem of finding a complex nonsingular matrix $P$ such that $P^{T}A_{j}P$ is as diagonal as possible, in some metric (see, for example, [3, 6, 26]).

In summary, the solution to the SDC problem provided extends earlier results from the 1980s, solves the initial motivating question for the authors related to evolution algebras and may have an impact on applications in optimisation or signal processing as described above.

In Section 2 we provide notation and definitions. In Section 3 we solve the SDC problem in the case of complex symmetric matrices and present a finite step procedure to determine whether a given set of matrices is SDC or not. In Section 4 we discuss possible avenues of further research.

2. Notation

Let $\mathcal{M}_{n,m}$ denote all $n\times m$ matrices over $\mathbb{C}$. Let $\mathcal{M}_{n}:=\mathcal{M}_{n,n}$, let $\mathcal{MS}_{n}$ be all symmetric elements in $\mathcal{M}_{n}$ and let $\mathcal{GL}_{n}$ be all invertible elements in $\mathcal{M}_{n}$. A diagonal matrix in $\mathcal{M}_{n}$ with diagonal entries $d_{1},\ldots,d_{n}$ will be written as D=\mathrm{\mathrm{diag}}\big{(}d_{1},\ldots,d_{n}\big{)}. For $A\in\mathcal{M}_{n}$ we denote its $ij$ component by $A_{ij}$ or \big{(}A\big{)}_{ij} and the zero and identity element in $\mathcal{M}_{n}$ are denoted $0_{n}$ and $I_{n}$, respectively. We recall that $A\in\mathcal{M}_{n}$ is said to be orthogonal if $A^{T}=A^{-1}$ (where $A^{T}$ denotes the usual transpose of $A$) and is said to be unitary if $\bar{A}^{T}=A^{-1}$ (where $\bar{A}$ denotes the entrywise complex conjugate of $A$); matrices $A,B\in\mathcal{M}_{n}$ are said to be congruent if there exists $P\in\mathcal{GL}_{n}$ such that $P^{T}AP=B$ and are said to be similar if there exists $P\in\mathcal{GL}_{n}$ such that $P^{-1}AP=B$. Congruent (or similar) matrices have the same rank. In fact, $A$ and $B$ are congruent if, and only if, they have the same rank [12, Theorem 4.5.12], hence $A$ similar to $B$ implies $A$ congruent to $B$ but the converse does not hold. We introduce the following definitions for a set of matrices in $\mathcal{M}_{n}$.

Definition 1**.**

Let $A_{1},\ldots,A_{m}\in\mathcal{M}_{n}$. We say $A_{1},\ldots,A_{m}$ are simultaneously diagonalizable via congruence (SDC for short) if there exists $P\in\mathcal{GL}_{n}$ and diagonal matrices $D_{1},\ldots,D_{m}\in\mathcal{M}_{n}$ such that

[TABLE]

Of course, if $A_{1},\ldots,A_{m}$ are SDC then they are necessarily symmetric.

Definition 2**.**

Let $L_{1},\ldots,L_{m}\in\mathcal{M}_{n}$. We say $L_{1},\ldots,L_{m}$ are simultaneously diagonalizable via similarity (SDS for short) if there exists $P\in\mathcal{GL}_{n}$ and diagonal matrices $D_{1},\ldots,D_{m}\in\mathcal{M}_{n}$ such that

[TABLE]

It is important to remark that even when $A_{1},\ldots,A_{m}$ in Definition 1 or $L_{1},\ldots,L_{m}$ in Definition 2 are real, the resulting matrices $P$ and $D_{j}$ may have to be complex, as illustrated in Example 16 below. The following result is well known (see for instance [12, Theorems 1.3.12 and 1.3.21]) and means that SDS is easy to check in practice, in contrast to SDC.

Theorem 3**.**

Let $L_{1},\ldots,L_{m}\in\mathcal{M}_{n}$. These matrices are simultaneously diagonalizable by similarity (SDS) if, and only if, they are all diagonalizable by similarity and they pairwise commute.

3. Solving the SDC problem

Let $S^{2m-1}:=\{x\in\mathbb{C}^{m}:\|x\|=1\}$, where $\left\|\cdot\right\|$ denotes the usual Euclidean norm. We use the standard concepts of linear pencil and maximum pencil rank.

Definition 4**.**

Let $A_{1},\ldots,A_{m}\in\mathcal{M}_{n}$. Define the associated linear pencil to be the map

[TABLE]

Since $\mathrm{rank}A\left(\lambda\right)=\mathrm{rank}A\left(\frac{\lambda}{\|\lambda\|}\right)$, for $\lambda\neq 0$, it follows that

[TABLE]

In addition, since $\{\mathrm{rank}{A}(\lambda):\lambda\in S^{2m-1}\}\subseteq\{0,1,\ldots,n\}$, it follows that the above supremum must be achieved. In other words, there exists some $\lambda_{0}\in S^{2m-1}$ such that

[TABLE]

Definition 5**.**

Let $A_{1},\ldots,A_{m}\in\mathcal{M}_{n}$. The rank of the associated linear pencil is $r:=\sup_{\begin{subarray}{c}\lambda\in\mathbb{C}^{m}\end{subarray}}\mathrm{rank}{A}(\lambda).$ We refer to $r$ as the (maximum pencil) rank of $A_{1},\ldots,A_{m}$ and denote it as $r=\mathrm{rank}(A_{1},\ldots,A_{m}).$ In the case that $r=n$ we say that the pencil is non-singular. From above, $r=\mathrm{rank}(A_{1},\ldots,A_{m})=\mathrm{rank}{A}(\lambda_{0})$, for some $\lambda_{0}\in S^{2m-1}$.

The following simple lemma is important.

Lemma 6**.**

Let $A_{1},\ldots,A_{m}\in\mathcal{M}_{n}$ and let $r=\mathrm{rank}(A_{1},\ldots,A_{m})=\mathrm{rank}A(\lambda_{0})$, for some $\lambda_{0}\in S^{2m-1}$. Then

[TABLE]

Proof.

Clearly $\bigcap_{j=1}^{m}\mathrm{ker}A_{j}\subseteq\mathrm{ker}{A}(\lambda)$ and hence

[TABLE]

In particular, for maximum pencil rank $r=\mathrm{rank}A(\lambda_{0})$ we have

$\bigcap_{j=1}^{m}\mathrm{ker}A_{j}\subseteq\mathrm{ker}{A}(\lambda_{0})$,  and  $\dim(\mathrm{ker}{A}(\lambda_{0}))=n-r$ then gives the result. ∎

We will see later that \dim\big{(}\bigcap_{j=1}^{m}\mathrm{ker}A_{j}\big{)}=n-r is necessary for $A_{1},\ldots,A_{m}$ to be SDC and, in this case, it follows from the above that the subspace $\mathrm{ker}{A}(\lambda_{0})$ is actually independent of the point $\lambda_{0}$ satisfying $r=\mathrm{rank}A(\lambda_{0})$.

3.1. The SDC problem for $n\times n$ matrices with non-singular pencils

We now solve the SDC problem for symmetric matrices $A_{1},\ldots,A_{m}\in\mathcal{M}_{n}$, in the particular case that $\mathrm{rank}(A_{1},\ldots,A_{m})=n$. The proof follows ideas from [12, Theorem 4.5.17], [14, Lemma 1] and [22, p.230]. In particular, the simple observation that if $A(\lambda)$ is invertible then

[TABLE]

motivates our first main result and proves it in the obvious direction.

Theorem 7**.**

Let ${A}_{1},\ldots,{A}_{m}\in\mathcal{MS}_{n}$ have maximum pencil rank $n$. For any $\lambda_{0}\in\mathbb{C}^{m}$ with $\mathrm{rank}A(\lambda_{0})=n$ then

[TABLE]

Proof.

Let $\lambda_{0}\in\mathbb{C}^{m}$ satisfy $\mathrm{rank}A(\lambda_{0})=n$.

In the forward direction, we assume that $A_{1},\ldots,A_{m}$ are SDC and let $P\in\mathcal{GL}_{n}$ satisfy $P^{T}A_{j}P$ is diagonal, for $j=1,\ldots,m$. Then $\ P^{T}A(\lambda_{0})P$ is diagonal and invertible giving

[TABLE]

is diagonal, for $1\leq j\leq m$ and we are done.

In the opposite direction, assume that $A(\lambda_{0})^{-1}A_{1},\ldots,A(\lambda_{0})^{-1}A_{m}$ are SDS and let $P\in\mathcal{GL}_{n}$ satisfy $D^{(j)}:=P^{-1}A(\lambda_{0})^{-1}A_{j}P$ is diagonal, for $j=1,\ldots,m$. We define symmetric matrices $B_{j}:=P^{T}A_{j}P$ and $B(\lambda_{0}):=P^{T}A(\lambda_{0})P$, to give

[TABLE]

Taking the transpose of this latter equation implies $B(\lambda_{0})$ commutes with $D^{(j)}$, for $j=1,\ldots,m$. Component-wise, this means that for all $1\leq k,l\leq n,$

[TABLE]

In particular, for all $1\leq k,l\leq n,$

[TABLE]

Write $D^{(j)}=\mathrm{diag}(\alpha^{j}_{1},\ldots,\alpha^{j}_{n})$, for $1\leq j\leq m$, and let $p_{j}$ satisfy

[TABLE]

($p_{j}$ is the length of the first run of identical diagonals in $D^{(j)}$) and define $n_{1}:=\min_{1\leq j\leq m}p_{j}.$ Define $\alpha^{(j)}_{1}:=\alpha^{j}_{1}$ and $\alpha^{(j)}_{2}:=\alpha^{j}_{n_{1}+1}$ so that

[TABLE]

and we may write

[TABLE]

and there is some $j\in\{1,\ldots,m\}$ for which $\alpha^{(j)}_{1}\neq\alpha^{(j)}_{2}$ ($I_{n}$ denotes the $n\times n$ identity matrix). We repeat a similar process twice more on $\mathrm{diag}(\alpha^{(j)}_{2},\alpha^{j}_{n_{1}+2},\ldots,\alpha^{j}_{n})$ to find $n_{2}$, $n_{3}$ and $\alpha^{(j)}_{3}:=\alpha^{j}_{n_{1}+n_{2}+1}$ so that, $\hbox{for all}\ 1\leq j\leq m$,

[TABLE]

while there is some $j$ for which $\alpha^{(j)}_{1}\neq\alpha^{(j)}_{2}$ and some $k$ for which $\alpha^{(k)}_{2}\neq\alpha^{(k)}_{3}$.

If now $\alpha^{(j)}_{1}=\alpha^{(j)}_{3}$, for all $1\leq j\leq m$, then we may re-order the diagonal entries to amalgamate $\alpha_{1}^{(j)}I_{n_{1}}$ and $\alpha_{3}^{(j)}I_{n_{3}}$, namely, there is an orthogonal permutation matrix $R\in\mathcal{GL}_{n}$ with

[TABLE]

$\hbox{ for all}\ 1\leq j\leq m.$

We continue this process of finding $\alpha_{l}^{(j)}$s for $\mathrm{diag}(\alpha^{j}_{n_{1}+n_{2}+n_{3}+1},\ldots,\alpha^{j}_{n})$. We amalgamate, as described above, where necessary so that for some orthogonal $U\in\mathcal{M}_{n}$ and $\text{ all}\ 1\leq j\leq m$ we have

[TABLE]

$\ \text{subject to the condition that for}\ 1\leq a<b\leq d,\ \text{there is some}\ j\in\{1,\ldots,m\}$ with $\alpha_{a}^{(j)}\neq\alpha_{b}^{(j)}.$ Of course, $d\leq n$ and $d$ is as small as possible satisfying the above.

We now write $U^{T}B(\lambda_{0})U$ as a $d\times d$ block matrix, whose $(a,b)$ sub-block, denoted here $[U^{T}B(\lambda_{0})U]_{ab},$ is of size $n_{a}\times n_{b}$, for $1\leq a,b\leq d$. Then $U^{T}B(\lambda_{0})U$ commutes with $U^{T}D^{(j)}U$, for all $1\leq j\leq m$, since $B(\lambda_{0})$ commutes with $D^{(j)}$, for all $1\leq j\leq m$, and $U$ is orthogonal. This commutativity then yields a block version of (3.3).

Specifically, since

[TABLE]

and we have from (3.4) that if $a<b$ then $\alpha_{a}^{(j)}\neq\alpha_{b}^{(j)}$, we get

[TABLE]

In other words, we have a block diagonal decomposition

[TABLE]

where $C_{a}\in\mathcal{MS}_{n_{a}},\,\,a=1,\ldots,d$.

As each $C_{a}$ must be symmetric, we can diagonalize it via a unitary transformation, as in [12, Corollary 2.6.6 (a)]. In other words, for each $a=1,\ldots,d$ there exists $V_{a}\in\mathcal{GL}_{n_{a}}$ unitary and $D_{a}$ a non-negative diagonal matrix such that

[TABLE]

We recall also that the diagonal entries of $D_{a}$ are the singular values of $C_{a}.$

Defining $V:=V_{1}\oplus\ldots\oplus V_{d}$ and $D:=D_{1}\oplus\ldots\oplus D_{d}$ (diagonal) then (3.6) and (3.7) give

[TABLE]

Defining now $Q=PUV$, and since $B(\lambda_{0})=P^{T}A(\lambda_{0})P$, (3.8) implies $Q^{T}A(\lambda_{0})Q=D$. In addition,

[TABLE]

which is clearly diagonal for all $j=1,\ldots,m$. ∎

We recall from Theorem 3 that $A(\lambda_{0})^{-1}A_{1},\ldots,A(\lambda_{0})^{-1}A_{m}$ are SDS if, and only if, they are all diagonalizable by similarity and they pairwise commute. It follows now from Theorem 7 that property SDS of the matrices $A(\lambda_{0})^{-1}A_{1},\ldots,A(\lambda_{0})^{-1}A_{m}$ is independent of the particular $\lambda_{0}$ chosen.

3.2. The SDC problem for $n\times n$ matrices with arbitrary pencil rank

3.2.1. Preliminaries: Diagonal matrices.

Lemma 8**.**

Let $D_{1},\dots,D_{m}$ be diagonal matrices in $\mathcal{M}_{n}$, ${D}$ be the associated linear pencil and $r$ be its maximum pencil rank. Then the following hold:

1. (i)

$D_{1},\ldots,D_{m}$* have zeros in the same $(n-r)$ diagonal positions and*

[TABLE] 2. (ii)

There is an orthogonal $Q\in\mathcal{M}_{n}$ such that $Q^{T}D_{j}Q=\widetilde{D}_{j}\oplus 0_{n-r}$, where $\widetilde{D}_{j}\in\mathcal{M}_{r}$ is diagonal, $1\leq j\leq m$.

*Moreover the pencil $\widetilde{D}$ associated to matrices $\widetilde{D}_{1},\ldots,\widetilde{D}_{m}\in M_{r}$ is non-singular

(and if $\lambda_{0}$ satisfies $r=\mathrm{rank}D(\lambda_{0})$ then $\widetilde{D}(\lambda_{0})\in\mathcal{GL}_{r}$).*

Proof.

Since $D$ has maximum pencil rank $r$, we choose $\lambda_{0}\in S^{2m-1}$ with $r=\mathrm{rank}{D}(\lambda_{0}).$ Writing D_{j}=\mathrm{\mathrm{diag}}\big{(}d^{j}_{1},\ldots,d^{j}_{n}\big{)}\in\mathcal{M}_{n}, $1\leq j\leq m$, we define vectors $u_{i}=\begin{pmatrix}d^{1}_{i}\\ \vdots\\ d^{m}_{i}\end{pmatrix}\in\mathbb{C}^{m}$, for $1\leq i\leq n.$ By direct calculation

[TABLE]

where $\cdot$ represents the dot product on $\mathbb{C}^{m}$ given by $\begin{pmatrix}z_{1}\\ \vdots\\ z_{m}\end{pmatrix}\cdot\begin{pmatrix}w_{1}\\ \vdots\\ w_{m}\end{pmatrix}=\sum_{i=1}^{m}z_{i}w_{i}.$

Since $r=\mathrm{rank}{D}(\lambda_{0}),$ we can then assume without loss of generality (up to rearrangement of the basis vectors) that $\lambda_{0}\cdot u_{i}\neq 0\,,\,\,\,1\leq i\leq r$ and $\lambda_{0}\cdot u_{j}=0\,,\,\,\,r+1\leq j\leq n$. In particular, $u_{i}\neq 0$, for $1\leq i\leq r.$

Define $h:\mathbb{C}^{m}\longrightarrow\mathbb{C}$ by $h(\lambda)=\Pi_{i=1}^{r}\lambda\cdot u_{i}$. Since $h$ is continuous, the set $A:=h^{-1}(\mathbb{C}\setminus\{0\})$ is open in $\mathbb{C}^{m}$ and since $\lambda_{0}\in A$, we have that for some $s>0$, $\lambda_{0}+v\in A$ and hence $h(\lambda_{0}+v)\neq 0$, for all $v\in\mathbb{C}^{m},\,\|v\|<s$. This gives $\mathrm{rank}{D}(\lambda_{0}+v)\geq r$ and since $r$ is the maximum rank of ${D}(\lambda)$, it follows that $\mathrm{rank}{D}(\lambda_{0}+v)=r$ and therefore $(\lambda_{0}+v)\cdot u_{j}=0$, for all $j$ with $r+1\leq j\leq n$. Thus $v\cdot u_{j}=0$, for all $r+1\leq j\leq n$, and all $v\in\mathbb{C}^{m},\,\|v\|<s$. This is impossible unless $u_{j}=0$, for all $r+1\leq j\leq n$ (otherwise $v=\overline{u}_{j}(\frac{s}{2\|u_{j}\|})$ will give a contradiction). In other words

[TABLE]

namely, $D_{1},\ldots,D_{m}$ have zeros in the same $n-r$ diagonal positions. It follows that \dim\big{(}\bigcap_{j=1}^{m}\mathrm{ker}D_{j}\big{)}\geq n-r. On the other hand, Lemma 6 and (3.1) then imply \dim\big{(}\bigcap_{j=1}^{m}\mathrm{ker}D_{j}\big{)}=n-r and $\bigcap_{j=1}^{m}\mathrm{ker}D_{j}=\mathrm{ker}{D}(\lambda_{0}).$

(ii) For $\lambda_{0}\in S^{2m-1}$ with $r=\mathrm{rank}{D}(\lambda_{0})$, we see in the proof of (i) that there is an orthogonal (permutation) matrix $Q\in\mathcal{M}_{n}$ satisfying $Q^{T}D_{j}Q=\widetilde{D}_{j}\oplus 0_{n-r}$, where $\widetilde{D}_{j}\in\mathcal{M}_{r}$ is diagonal, $1\leq j\leq m$. Let $\widetilde{D}$ be the reduced linear pencil associated to $\widetilde{D}_{1},\ldots,\widetilde{D}_{m}.$ Then $Q^{T}D(\lambda_{0})Q=\widetilde{D}(\lambda_{0})\oplus 0_{n-r}$, so $\mathrm{rank}\widetilde{D}(\lambda_{0})=\mathrm{rank}D(\lambda_{0})=r$ and $\widetilde{D}(\lambda_{0})\in\mathcal{GL}_{r}.$ ∎

The next theorem enables us, when considering whether or not a set of $n\times n$ matrices is SDC, to reduce the problem to a set of $r\times r$ matrices, where $r$ is the maximum pencil rank.

Theorem 9**.**

Let $A_{1},\ldots,A_{m}\in\mathcal{MS}_{n}$ have maximum pencil rank $r$. Then

[TABLE]

[TABLE]

Moreover, if either of the above conditions is satisfied, the pencil $\widetilde{D}$ associated to matrices $\widetilde{D}_{1},\ldots,\widetilde{D}_{m}\in M_{r}$ is non-singular (and if $\lambda_{0}$ satisfies $r=\mathrm{rank}A(\lambda_{0})$ then $\widetilde{D}(\lambda_{0})\in\mathcal{GL}_{r}$).

Proof.

Let $A_{1},\ldots,A_{m}\in\mathcal{MS}_{n}$ have maximum pencil rank $r$. Choose any $\lambda_{0}\in S^{2m-1}$ satisfying $r=\mathrm{rank}A(\lambda_{0})$.

For the forward direction, assume that $A_{1},\ldots,A_{m}$ are SDC. Then there exists $S\in\mathcal{GL}_{n}$ and diagonal matrices $D_{1},\ldots,D_{m}$ such that

[TABLE]

Let $D$ be the pencil associated to matrices $D_{1},\ldots,D_{m}$. Then $S^{T}{A}(\lambda)S={D}(\lambda),$ for all $\lambda\in\mathbb{C}^{m},$ so maximum pencil ranks for $A(\lambda)$ and $D(\lambda)$ agree and $r=\mathrm{rank}A(\lambda_{0})=\mathrm{rank}D(\lambda_{0}).$ From Lemma 8 (ii) there then exists an orthogonal $Q\in\mathcal{M}_{n}$ such that $Q^{T}D_{j}Q=\widetilde{D}_{j}\oplus 0_{n-r}$, where $\widetilde{D}_{j}\in\mathcal{M}_{r}$ is diagonal, $1\leq j\leq m$ and $\widetilde{D}(\lambda_{0})\in\mathcal{GL}_{r}$. Then $P=SQ$ gives

[TABLE]

for $1\leq j\leq m$ as desired and for pencil $\widetilde{D}$ associated to matrices $\widetilde{D}_{1},\ldots,\widetilde{D}_{m}$ clearly $r=\mathrm{rank}\widetilde{D}(\lambda_{0})$.

Since $S^{T}{A}(\lambda_{0})S={D}(\lambda_{0})$ we have $\mathrm{ker}{A}(\lambda_{0})=S(\mathrm{ker}{D}(\lambda_{0}))$. On the other hand, from Lemma 8 (i), Lemma 6 and (3.9) we have $S(\mathrm{ker}{D}(\lambda_{0}))=S(\bigcap_{j=1}^{m}{\ker}D_{j})=\bigcap_{j=1}^{m}{\ker}A_{j}$. In other words $\bigcap_{j=1}^{m}{\ker}A_{j}=\mathrm{ker}{A}(\lambda_{0})$ has dimension $n-r$. The opposite direction is trivial.

∎

3.2.2. The general case of non-diagonal matrices with arbitrary pencil rank

The following Lemma holds regardless of diagonalizability and is key to solving the SDC problem in the general case.

Lemma 10**.**

Let $A_{1},\ldots,A_{m}\in\mathcal{MS}_{n}$ have maximum pencil rank $r.$ Then

[TABLE]

there exists $Q\in\mathcal{GL}_{n}$ with

[TABLE]

Moreover, if either of the above conditions is satisfied, the pencil $\widetilde{A}$ associated to matrices $\widetilde{A}_{1},\ldots,\widetilde{A}_{m}\in M_{r}$ is non-singular (and if $\lambda_{0}$ satisfies $r=\mathrm{rank}A(\lambda_{0})$ then $\widetilde{A}(\lambda_{0})\in\mathcal{GL}_{r}$).

Proof.

Let $A_{1},\ldots,A_{m}\in\mathcal{MS}_{n}$ have maximum pencil rank $r$.

In the forward direction, assume that $\mathcal{V}:=\bigcap_{j=1}^{m}\mathrm{ker}A_{j}$ has $\dim(\mathcal{V})=n-r.$ Choose a basis $v_{r+1},\ldots,v_{n}$ of $\mathcal{V}$ and extend by vectors $v_{1},\ldots,v_{r}$ to get a basis $v_{1},\ldots,v_{n}$ of $\mathbb{C}^{n}.$ Let $Q\in\mathcal{GL}_{n}$ be the matrix whose $i$th column is given by the vector $v_{i}$. For $r+1\leq i\leq n$, we have $v_{i}\in\ker A_{j}$ and hence $Q^{T}A_{j}Q(e_{i})=Q^{T}A_{j}(v_{i})=0$, for all $1\leq j\leq m$ (and $e_{i}$ is the column vector with $1$ in the $i$th position and all other entires [math]). In other words, columns $r+1$ to $n$ of $Q^{T}A_{j}Q$ are identically zero and, since $Q^{T}A_{j}Q$ is symmetric, it follows that

[TABLE]

as desired. In the opposite direction, assume that (3.10) holds for some $Q\in\mathcal{GL}_{n}$. Then Q\big{(}0_{r}\oplus\mathbb{C}^{n-r}\big{)}\subseteq\bigcap_{j=1}^{m}\mathrm{ker}A_{j} so $\dim(\bigcap_{j=1}^{m}\mathrm{ker}A_{j})\geq n-r$. Equality now follows from (3.1).

Finally, if the conditions in the statement hold, the reduced pencil $\widetilde{A}$ associated with $\widetilde{A}_{1},\ldots,\widetilde{A}_{m}\in M_{r}$ has maximum pencil rank $r$, since for any $\lambda_{0}\in\mathbb{C}^{m}$ with $r=\mathrm{rank}A(\lambda_{0})$ then (3.11) implies that $\mathrm{rank}\widetilde{A}(\lambda_{0})=r$ and $\widetilde{A}(\lambda_{0})\in\mathcal{GL}_{r}.$ ∎

Remarks 11**.**

We note that by choosing the basis vectors $v_{1},\ldots,v_{n}$ in the above proof to be orthogonal, with respect to the complex inner product $<z,w>:=z\cdot\overline{w}$ on $\mathbb{C}^{n}$, $Q$ can be chosen to be unitary.

Lemma 10 therefore allows us to find matrices $\widetilde{A}_{1},\ldots,\widetilde{A}_{m}$ satisfying (3.10) using only the kernels of the $A_{j}$. This enables us, subject to the condition that \dim\big{(}\bigcap_{j=1}^{m}\mathrm{ker}A_{j}\big{)}=n-r, to reduce the dimension of the problem by proving that

[TABLE]

Theorem 7 then motivates the following definition.

Definition 12** (Reduced maximal-rank matrices).**

Let $A_{1},\ldots,A_{m}\in\mathcal{MS}_{n}$ have maximum pencil rank $r$ and satisfy $\dim(\bigcap_{j=1}^{m}\mathrm{ker}A_{j})=n-r$. Let $\widetilde{A}_{1},\ldots,\widetilde{A}_{m}$ be as in (3.10) and fix $\lambda_{0}\in S^{2m-1}$ with $r=rankA(\lambda_{0})$. Reduced pencil $\widetilde{A}$ then has $\widetilde{A}(\lambda_{0})\in\mathcal{GL}_{r}$.

We define the $r\times r$ matrices

[TABLE]

Remarks 13**.**

$L_{1},\ldots,L_{m}$* are not symmetric in general and $\sum_{j=1}^{m}(\lambda_{0})_{j}L_{j}=I_{n}$. In addition, Theorem 7 states that $L_{1},\ldots,L_{m}$ are SDS if, and only if, $\widetilde{A}_{1},\ldots,\widetilde{A}_{m}$ are SDC and consequently the condition is independent of the particular $\lambda_{0}$ chosen in the definition. For this reason we write $L_{j}$ instead of $L_{j}(\lambda_{0})$ .*

The following is our main theorem.

Theorem 14**.**

Let $A_{1},\ldots,A_{m}\in\mathcal{MS}_{n}$ have maximum pencil rank $r$. Then

[TABLE]

where $L_{1},\ldots,L_{m}$ are as in Definition 12 above.

Proof.

Let $A_{1},\ldots,A_{m}\in\mathcal{MS}_{n}$ have maximum pencil rank $r$ and choose $\lambda_{0}\in S^{2m-1}$ satisfying $r=rankA(\lambda_{0})$.

In the forward direction, assume now that $A_{1},\ldots,A_{m}$ are SDC.

From Theorem 9 $\ \dim(\bigcap_{j=1}^{m}\mathrm{ker}A_{j})=n-r$ and there exists $P\in\mathcal{GL}_{n}$ such that

[TABLE]

In addition, if $\widetilde{D}$ is the pencil associated to $r\times r$ matrices $\widetilde{D}_{1},\ldots,\widetilde{D}_{m}$ then $\widetilde{D}(\lambda_{0})\in\mathcal{GL}_{r}$. Lemma 10 then gives $Q\in\mathcal{GL}_{n}$ with

[TABLE]

and $\widetilde{A}(\lambda_{0})\in\mathcal{GL}_{r}$, for reduced pencil $\widetilde{A}(\lambda_{0})=\sum_{j=1}^{m}(\lambda_{0})_{j}\widetilde{A}_{j}$.

Thus for $R=Q^{-1}P$ (3.13) and (3.14) give

[TABLE]

Writing $R$ as a block matrix,

[TABLE]

for $S\in\mathcal{M}_{r},V\in\mathcal{M}_{n-r},U\in\mathcal{M}_{n-r,r},T\in\mathcal{M}_{r,n-r}$, it follows from (3.15) and matrix multiplication that

[TABLE]

Then for reduced $r\times r$ matrix pencils (and $\lambda=(\lambda_{1},\ldots,\lambda_{m})\in\mathbb{C}^{m}$)

[TABLE]

(3.16) gives

[TABLE]

and, in particular,

[TABLE]

Since $\widetilde{A}(\lambda_{0})$ and $\widetilde{D}(\lambda_{0})$ are invertible, it follows that $S$ is invertible and combining (3.16) and (3.18) gives

[TABLE]

In particular, $S^{-1}{\widetilde{A}(\lambda_{0})}^{-1}\widetilde{A}_{j}S$ are diagonal for all $j=1,\ldots,m$. In other words the $r\times r$ matrices

[TABLE]

are SDS and we are done.

For the opposite direction, let us assume that $\dim(\bigcap_{j=1}^{m}\mathrm{ker}A_{j})=n-r$ and that $L_{1},\ldots,L_{m}\ \text{ are SDS}$. Then from Lemma 10 there exists $Q\in\mathcal{GL}_{n}$ such that

[TABLE]

with $\widetilde{A}_{j}\in\mathcal{MS}_{r}$ and $\widetilde{A}(\lambda_{0})\in\mathcal{GL}_{r}\cap\mathcal{MS}_{r}$. Construct $L_{j}={\widetilde{A}(\lambda_{0})}^{-1}\widetilde{A}_{j},\quad j=1,\ldots,m$ as in Definition 12 above. By hypothesis, these matrices are SDS so from Theorem 7 it follows that $\widetilde{A}_{1},\ldots\widetilde{A}_{m}$ are SDC, namely, there exists $P\in\mathcal{GL}_{r}$ such that $P^{T}\widetilde{A}_{j}P=D_{j}$, for $D_{j}$ diagonal in $\mathcal{M}_{r}$, for all $j=1,\ldots,m$. Define $R:=P\oplus I_{n-r}\in\mathcal{GL}_{n}$. Then

[TABLE]

Thus, for $S=QR\in\mathcal{GL}_{n}$ we have

[TABLE]

diagonal for all $j=1,\ldots,m$. Thus, $A_{1},\ldots,A_{m}$ are SDC. ∎

3.3. A procedure to solve the SDC problem

The above results allow us now to determine in a finite number of steps whether or not a set of matrices are SDC. Given $A_{1},\ldots,A_{m}\in\mathcal{MS}_{n}$, let r:=\mathrm{rank}\big{(}A_{1},\ldots,A_{m}\big{)}. From (3.1), we have

[TABLE]

and Theorems 14 and 3 now give us the following procedure.

1. (1)

If $\ \dim(\bigcap_{j=1}^{m}\mathrm{ker}A_{j})<n-r$ then $A_{1},\ldots,A_{m}$ are not SDC. 2. (2)

If $\ \dim(\bigcap_{j=1}^{m}\mathrm{ker}A_{j})=n-r$ then we calculate $L_{1},\ldots,L_{m}$ from Eq. (3.12), for some $\lambda_{0}\in\mathbb{C}^{m}$ with $r=\mathrm{rank}A(\lambda_{0})$. If $L_{1},\ldots,L_{m}$ do not pairwise commute then $A_{1},\ldots,A_{m}$ are not SDC. 3. (3)

If $\ \dim(\bigcap_{j=1}^{m}\mathrm{ker}A_{j})=n-r$, $L_{1},\ldots,L_{m}$ do pairwise commute, and if each $L_{1},\ldots,L_{m}$ is diagonalizable by similarity then $A_{1},\ldots,A_{m}$ are SDC. Otherwise (that is, if any one $L_{j}$ is not diagonalizable by similarity) $A_{1},\ldots,A_{m}$ are not SDC.

Remarks 15**.**

Regarding (1) above, we note that estimation may be sufficient to determine if $\ \dim(\bigcap_{j=1}^{m}\mathrm{ker}A_{j})<n-r$, as in Example 17 below. Regarding (2), since $\sum_{j=1}^{m}(\lambda_{0})_{j}L_{j}=I_{n}$, it suffices to check if $m-1$ of $L_{1},\ldots,L_{m}$ pairwise commute. Regarding (3), we recall that $L_{i}$ are not symmetric in general.

Example 16** ($n=2,m=2$).**

Let A_{1}=\left(\begin{array}[c]{cc}0&1\\ 1&1\end{array}\right),\,\,A_{2}=\left(\begin{array}[c]{cc}1&1\\ 1&0\end{array}\right)\,. We apply the above procedure.

1. (1)

$\mathrm{ker}{A_{1}}=\mathrm{ker}{A_{2}}=\{0\}$* so $\dim(\bigcap_{j=1}^{2}\mathrm{ker}A_{j})=0.$ As $A_{1}$ is nonsingular we take $\lambda_{0}=(1,0)$ so $A(\lambda_{0})=A_{1}$ and $\mathrm{rank}{A_{1}}=2$. Therefore $r=2$ and $\ \dim(\bigcap_{j=1}^{m}\mathrm{ker}A_{j})=n-r$ holds and we continue to the next step.* 2. (2)

We compute $L_{1}=A_{1}^{-1}A_{1}=I_{2}$ and L_{2}=A_{1}^{-1}A_{2}=\left(\begin{array}[c]{cc}0&-1\\ 1&1\end{array}\right). These matrices (trivially) commute so we continue to the next step. 3. (3)

$L_{1}$* is diagonal. $L_{2}$ is diagonalizable by similarity as it has $2$ different eigenvalues: $d_{\pm}=(1\pm i\sqrt{3})/2$. Therefore the matrices $A_{1},A_{2}$ are SDC.*

Explicitly, $P^{-1}L_{2}P=\mathrm{\mathrm{diag}}(d_{+},d_{-})$ with P=\left(\begin{array}[c]{cc}ad_{-}&bd_{+}\\ -a&-b\end{array}\right) for any $a,b\in\mathbb{C}$ with $ab\neq 0$. Note that $P$ cannot be made real by any choice of the constants $a,b$. We have, finally,

[TABLE]

Example 17** ($n=3,m=2$).**

Let A_{1}=\left(\begin{array}[c]{ccc}1&1&0\\ 1&0&0\\ 0&0&0\end{array}\right),\,\,\,A_{2}=\left(\begin{array}[c]{ccc}0&0&1\\ 0&0&0\\ 1&0&0\end{array}\right)\,.

We apply the above procedure.

1. (1)

We calculate $\mathrm{ker}A_{1}=\mathrm{span}\{(0,0,1)^{T}\}$ and $\mathrm{ker}A_{2}=\mathrm{span}\{(0,1,0)^{T}\}$. Thus $\mathrm{ker}{A_{1}}\cap\mathrm{ker}{A_{2}}=\{0\}$ so $\dim(\bigcap_{j=1}^{2}\mathrm{ker}A_{j})=0.$

Since

[TABLE]

we have $r\leq 2$ and then $n-r\geq 1$. Therefore $\ \dim(\bigcap_{j=1}^{2}\mathrm{ker}A_{j})<n-r$ and hence $A_{1},A_{2}$ are not SDC.

4. Discussion

In this paper we solved the long-standing problem of simultaneous diagonalization via congruence in the complex symmetric case, providing also an explicit set of steps to solve this problem. The complex case has applications in signal processing, in particular to the problem of blind source separation. This latter problem is based on the exact SDC problem but, due to experimental and numerical errors in obtaining the target matrices $A_{1},\ldots,A_{m}$, it relies on the so-called approximate joint diagonalization, which is an optimisation problem. Our results could shed light on these approximate problems, as these problems usually consider an ad-hoc cost function [3, 26], which does not take into account the kernels of the target matrices.

Some optimisation-related applications consider the special case where, in the context of our Definition 1, the symmetric matrices $A_{1},\ldots,A_{m}$ are real, and the corresponding transformation matrix $P$ and resulting diagonal matrices $D_{1},\ldots,D_{m}$ are required to be real. In the context of Theorem 14 above, such a case would impose extra conditions of realness on the eigenvectors and eigenvalues of the reduced matrices $L_{1},\ldots,L_{m}$.

In many applications in genetics the matrices $L_{1},\ldots,L_{m}$ turn out to commute, but may not necessarily be diagonalizable. Thus, the SDC problem could be relaxed to a weaker problem, namely that of simultaneous block diagonalization [21].

Further research on building an algorithm to solve the SDC problem will focus on developing an efficient method for finding $\lambda_{0}$ such that the pencil $A(\lambda_{0})$ has maximum rank.

Acknowledgments The authors thank the referees for several helpful comments and suggestions and for bringing key references to our attention.