Spectral Analysis for Non-Hermitian Matrices and Directed Graphs

TL;DR

This paper extends spectral graph theory to non-Hermitian matrices and directed graphs, introducing new conditions and bounds that broaden the applicability of spectral methods.

Contribution

It introduces admissibility conditions and variational estimates for non-Hermitian matrices, enabling spectral analysis of directed graphs.

Findings

New bounds on independent set size in directed graphs

Generalization of spectral estimates to non-Hermitian matrices

Applicable techniques for analyzing non-Hermitian systems

Abstract

We generalize classical results in spectral graph theory and linear algebra more broadly, from the case where the underlying matrix is Hermitian to the case where it is non-Hermitian. New admissibility conditions are introduced to replace the Hermiticity condition. We prove new variational estimates of the Rayleigh quotient for non-Hermitian matrices. As an application, a new Delsarte-Hoffman-type bound on the size of the largest independent set in a directed graph is developed. Our techniques consist in quantifying the impact of breaking the Hermitian symmetry of a matrix and are broadly applicable.

Equations

$$\forall \mathbf{f}\in {\mathbb{C}}^{n\times 1},\mathbf{f}\ne 0,\min \left\{\mathbf{\lambda }(\mathbf{B})\right\}\le \frac{{\mathbf{f}}^{\ast }\mathbf{Bf}}{\Vert \mathbf{f}{\Vert }_2^2}\le \max \left\{\mathbf{\lambda }(\mathbf{B})\right\}.$$

$$\mathbf{f}=\mathbf{V}\overline{\mathbf{F}},\mathbf{g}=\mathbf{UG}\text{mbox}s.t.\forall 0\le \ell <n,\left(\mathbf{F}\left[\ell \right]\mathbf{G}\left[\ell \right]e^{i{\theta }_{\ell }}\right)\in \mathbb{R}.$$

$$\underset{0\le \ell <n}{\min }\left\{\left|{\lambda }_{\ell }(\mathbf{B})\right|\mathrm{sgn}(\mathbf{F}\left[\ell \right]\mathbf{G}\left[\ell \right]e^{i{\theta }_{\ell }})\right\}\le \frac{{\mathbf{f}}^{\ast }\mathbf{Bg}}{\sum _{0\le \ell <n}\left|\mathbf{F}\left[\ell \right]\mathbf{G}\left[\ell \right]\right|}\le \underset{0\le \ell <n}{\max }\left\{\left|{\lambda }_{\ell }(\mathbf{B})\right|\mathrm{sgn}(\mathbf{F}\left[\ell \right]\mathbf{G}\left[\ell \right]e^{i{\theta }_{\ell }})\right\}.$$

$${\mathbf{f}}^{\ast }\mathbf{Bg}=$$

$$=$$

$$\underset{0\le \ell <n}{\min }\left\{\left|{\lambda }_{\ell }(\mathbf{B})\right|\mathrm{sgn}(\mathbf{F}\left[\ell \right]\mathbf{G}\left[\ell \right]e^{i{\theta }_{\ell }})\right\}\sum _{0\le \ell <n}\left|\mathbf{F}\left[\ell \right]\mathbf{G}\left[\ell \right]\right|$$

$$\le$$

$$\sum _{0\le \ell <n}\left|\mathbf{F}\left[\ell \right]\mathbf{G}\left[\ell \right]\right|=\sum _{0\le \ell <n}\mathbf{F}\left[\ell \right]\mathbf{G}\left[\ell \right]=⟨\mathbf{f},\mathbf{g}⟩$$

$$\sum _{0\le \ell <n}\left|\mathbf{F}\left[\ell \right]\mathbf{F}\left[\ell \right]\right|=\Vert \mathbf{f}{\Vert }_2^2,$$

$$\underset{0\le \ell <n}{\min }\left|{\lambda }_{\ell }(\mathbf{B})\right|\mathrm{sgn}(\mathbf{F}\left[\ell \right]\mathbf{F}\left[\ell \right]e^{i{\theta }_{\ell }})=\min \{\mathbf{\lambda }(\mathbf{B})\},$$

$$\underset{0\le \ell <n}{\max }\left|{\lambda }_{\ell }(\mathbf{B})\right|\mathrm{sgn}(\mathbf{F}\left[\ell \right]\mathbf{F}\left[\ell \right]e^{i{\theta }_{\ell }})=\max \{\mathbf{\lambda }(\mathbf{B})\},$$

$$\mathbf{f}=\mathbf{U}\overline{\mathbf{F}},\mathbf{g}=\mathbf{VG}\text{mbox}s.t.\forall 0\le \ell <n,\left(\mathbf{F}\left[\ell \right]\mathbf{G}\left[\ell \right]e^{i{\theta }_{\ell }}\right)\in \mathbb{R}.$$

$$\underset{0\le \ell <n}{\min }\left\{\left|{\sigma }_{\ell }(\mathbf{B})\right|\mathrm{sgn}(\mathbf{F}\left[\ell \right]\mathbf{G}\left[\ell \right]e^{i{\theta }_{\ell }})\right\}\le \frac{{\mathbf{f}}^{\ast }\mathbf{Bg}}{\sum _{0\le \ell <n}\left|\mathbf{F}\left[\ell \right]\mathbf{G}\left[\ell \right]\right|}\le \underset{0\le \ell <n}{\max }\left\{\left|{\sigma }_{\ell }(\mathbf{B})\right|\mathrm{sgn}(\mathbf{F}\left[\ell \right]\mathbf{G}\left[\ell \right]e^{i{\theta }_{\ell }})\right\}.$$

$${\mathbf{f}}^{\ast }\mathbf{Bg}=$$

$$=$$

$$\underset{0\le \ell <n}{\min }\left\{\left|{\sigma }_{\ell }(\mathbf{B})\right|\mathrm{sgn}(\mathbf{F}\left[\ell \right]\mathbf{G}\left[\ell \right]e^{i{\theta }_{\ell }})\right\}\sum _{0\le \ell <n}\left|\mathbf{F}\left[\ell \right]\mathbf{G}\left[\ell \right]\right|$$

$$\le$$

$$\left({\mathbf{A}}^{{\text{R}}_{\theta }}\right)\left[j,\ell \right]=$$

$$\begin{cases}\begin{array}[]{c}\mathbf{A}\left[\left(j-\left\lfloor\frac{n}{2}\right\rfloor\right)\cos\theta+\left(\left\lfloor\frac{n}{2}\right\rfloor-\ell\right)\sin\theta+\left\lfloor\frac{n}{2}\right\rfloor,\left(j-\left\lfloor\frac{n}{2}\right\rfloor\right)\sin\theta-\left(\left\lfloor\frac{n}{2}\right\rfloor-\ell\right)\cos\theta+\left\lfloor\frac{n}{2}\right\rfloor\right],\>n\text{ odd}\\ \\ \mathbf{A}\left[\left(j-\frac{n-1}{2}\right)\cos\theta+\left(\frac{n-1}{2}-\ell\right)\sin\theta+\frac{n-1}{2},\left(j-\frac{n-1}{2}\right)\sin\theta-\left(\frac{n-1}{2}-\ell\right)\cos\theta+\frac{n-1}{2}\right],\>n\text{ even}\end{array}\end{cases}$$

$$\mathbf{A}^{\text{R}_{0}}=\mathbf{A}=\left(\begin{array}[]{rrr}a_{00}&a_{01}&a_{02}\\ a_{10}&a_{11}&a_{12}\\ a_{20}&a_{21}&a_{22}\end{array}\right),$$

$$\mathbf{A}^{\text{R}_{\frac{\pi}{2}}}=\left(\begin{array}[]{rrr}a_{20}&a_{10}&a_{00}\\ a_{21}&a_{11}&a_{01}\\ a_{22}&a_{12}&a_{02}\end{array}\right),\mathbf{A}^{\text{R}_{\pi}}=\left(\begin{array}[]{rrr}a_{22}&a_{21}&a_{20}\\ a_{12}&a_{11}&a_{10}\\ a_{02}&a_{01}&a_{00}\end{array}\right),\mathbf{A}^{\text{R}_{\frac{3\pi}{2}}}=\left(\begin{array}[]{rrr}a_{02}&a_{12}&a_{22}\\ a_{01}&a_{11}&a_{21}\\ a_{00}&a_{10}&a_{20}\end{array}\right).$$

$${\mathbf{A}}^{\ast }\mathbf{A}=\mathbf{I}=\mathbf{A}{\mathbf{A}}^{\ast },$$

$$\quad\mathbf{A}^{\text{R}_{\frac{\pi}{2}}}\left[j,:\right]\left(\mathbf{A}^{\text{R}_{\frac{\pi}{2}}}\left[\ell,:\right]\right)^{*}=\mathbf{A}\left[n-1-j,:\right]\left(\mathbf{A}\left[n-1-\ell,:\right]\right)^{*}=\begin{cases}\begin{array}[]{cc}1,&j=\ell,\\ 0,&\text{otherwise},\end{array}\end{cases}$$

$$\left(\mathbf{AB}\right)\left[j,\ell \right]=\sum _{0\le t<n}\mathbf{A}\left[j,t\right]\mathbf{B}\left[t,\ell \right].$$

$${\left(\mathbf{AB}\right)}^{{\text{R}}_{\theta }}\left[j,\ell \right]=$$

$$\sum _{0\le t<n}\mathbf{A}\left[\left(j-\lfloor \frac{n}{2}\rfloor \right)\cos \theta +\left(\lfloor \frac{n}{2}\rfloor -\ell \right)\sin \theta +\lfloor \frac{n}{2}\rfloor ,t\right]\mathbf{B}\left[t,\left(j-\lfloor \frac{n}{2}\rfloor \right)\sin \theta -\left(\lfloor \frac{n}{2}\rfloor -\ell \right)\cos \theta +\lfloor \frac{n}{2}\rfloor \right].$$

$${\left(\mathbf{AB}\right)}^{{\text{R}}_{\frac{\pi }{2}}}\left[j,\ell \right]=\sum _{0\le t<n}\mathbf{B}\left[t,j\right]\mathbf{A}\left[\left(n-1\right)-\ell ,t\right].$$

$${\left({\mathbf{A}}^{\top }\right)}^{{\text{R}}_{\theta }}\left[j,\ell \right]=$$

$$\mathbf{A}\left[\left(\ell -\lfloor \frac{n}{2}\rfloor \right)\cos \left(-\theta \right)+\left(\lfloor \frac{n}{2}\rfloor -j\right)\sin \left(-\theta \right)+\lfloor \frac{n}{2}\rfloor ,\left(\ell -\lfloor \frac{n}{2}\rfloor \right)\sin \left(-\theta \right)-\left(\lfloor \frac{n}{2}\rfloor -j\right)\cos \left(-\theta \right)+\lfloor \frac{n}{2}\rfloor \right]$$

$${\left({\mathbf{A}}^{\top }\right)}^{{\text{R}}_{\frac{3\pi }{2}}}\left[j,\ell \right]=\mathbf{A}\left[\left(n-1\right)-j,\ell \right],$$

$${\mathbf{M}}^{{\text{R}}_{\frac{\pi }{2}}}=\mathbf{U}\mathrm{diag}\left\{\mathbf{\lambda }\left({\mathbf{M}}^{{\text{R}}_{\frac{\pi }{2}}}\right)\right\}{\mathbf{V}}^{\ast },\mathbf{U}{\mathbf{V}}^{\ast }=\mathbf{I}.$$

Full text

1 Introduction

The eigendecomposition is among the most powerful tools for analyzing Hermitian matrices $\mathbf{B}\in\mathbb{C}^{n\times n}$, i.e. matrices satisfying $\mathbf{B}^{*}=\mathbf{B}$, where $\mathbf{B}^{*}\left[j,\ell\right]=\overline{\mathbf{B}\left[\ell,j\right]}$. Several classical results in linear algebra can be derived from decomposing a Hermitian matrix $\mathbf{B}\in\mathbb{C}^{n\times n}$ as $\mathbf{B}=\mathbf{U}\operatorname{diag}(\boldsymbol{\lambda}(\mathbf{B}))\mathbf{U}^{*}$, where $\mathbf{U}$ is unitary and $\boldsymbol{\lambda}(\mathbf{B})=\left(\lambda_{\ell}(\mathbf{B})\right)_{0\leq\ell<n}\subset\mathbb{R}$. In particular, under the assumption that $\mathbf{B}$ is Hermitian, variational estimates on the Rayleigh quotient [HJ90] can be stated in terms of $\boldsymbol{\lambda}(\mathbf{B})$:

[TABLE]

In case that $\mathbf{B}$ is non-Hermitian, the eigenvalues of $\mathbf{B}$ may be complex, or even worse the eigendecomposition may not exist at all.

When $\mathbf{B}$ is non-Hermitian but diagonalizable, its eigendecomposition is of the form $\mathbf{B}=\mathbf{P}\operatorname{diag}(\boldsymbol{\lambda}(\mathbf{B}))\mathbf{P}^{-1}$ for some invertible matrix $\mathbf{P}$ and scalars $\boldsymbol{\lambda}(\mathbf{B})$. Compared to the Hermitian case, the columns of $\mathbf{P}$ need not form an orthonormal basis. A different decomposition that is available to all matrices is the singular value decomposition (SVD): $\mathbf{B}=\mathbf{U}\operatorname{diag}(\boldsymbol{\sigma}(\mathbf{B}))\mathbf{V}^{*}$, where the singular values $\boldsymbol{\sigma}(\mathbf{B})=\left(\sigma_{\ell}(\mathbf{B})\right)_{0\leq\ell<n}$ are non-negative and $\mathbf{U},\mathbf{V}$ are unitary matrices. However, it need not be the case that $\mathbf{U}\mathbf{V}^{*}=\mathbf{U}^{*}\mathbf{V}=\mathbf{I}$, which is the primary contrast with the eigendecomposition.

Powerful tools of linear algebra can be applied to the study of graphs via spectral graph theory [Chu97]. Indeed, let $\mathcal{G}=(V,\mathbf{B})$ be a graph, where $V$ is the set of vertices and $\mathbf{B}\in\{0,1\}^{|V|\times|V|}$ an adjacency matrix such that $\mathbf{B}\left[j,\ell\right]=1$ if there is an edge between the $j^{th}$ and $\ell^{th}$ nodes. By analyzing the spectral properties of $\mathbf{B}$, a variety of mathematical ideas may be adapted to $\mathcal{G}$, including notions of geometry [Moh89, MACO91], Fourier and wavelet analysis [CM06, HVG11, SRV16], random diffusion processes [CLL*+*05, CL06], and clusters [SM00, NJW02]. While these tools have contributed to a renaissance in the analysis of data, spectral graph methods almost uniformly require the underlying graph $\mathcal{G}$ to be undirected, or in linear algebra terms, $\mathbf{B}$ must be Hermitian. This is a severe limitation in practice, as a variety of real data does not lend itself to representation as an undirected graph, for example social networks [KLPM10], models for the spread of contagious disease in a heterogenous population [KW92], and predator-prey relationships [YA73].

1.1 Summary of Contributions

This article develops new approaches for the analysis of non-Hermitian matrices. The primary contributions are twofold. First, we prove a generalized version of the classical variational estimates on the Rayleigh quotient. New admissibility conditions are introduced to replace the Hermiticity condition. Second, the Delsarte-Hoffman bound on the size of independent sets in undirected graphs is generalized to the directed setting. Our major tool consists in quantifying the discrepancy between Hermitian and non-Hermitian matrices, and proposing additional hypotheses in the theorems to address this discrepancy. The gap between Hermiticity and non-Hermiticity is made precise, and moreover in the case of the Delsarte-Hoffman bound, the gaps between $\mathbf{B}$ being Hermitian, diagonalizable (with potentially complex eigenvalues) and $\mathbf{B}$ arbitrary is considered by analyzing the SVD. Our proof methods are flexible, and may be applicable to settings not considered in the present article.

1.2 Related Work

Spectral graph theory has been attempted for operators defined on directed graphs in a variety of contexts, including for the graph Laplacian [AC05, Chu05, But07, Bau12, BS13, ZS16, FAFS18, CMZ18] and for nonreversible Markov chains [Fil91]. Combinatorial results for directed graphs have also been studied [Bru10, KS15]. These results do not, however, develop precise characterizations of the ways in which classical results can be modified in the non-Hermitian setting. In particular, the admissibility conditions proposed in this article explicitly illustrate what is lost when a matrix is perturbed to deviate from Hermiticity, and suggest how to compensate for the loss of Hermiticity. Moreover, the proposed generalized Delsarte-Hoffman bound makes no assumptions of normality of the adjacency matrix of the underlying graph.

1.3 Notation

Throughout, bold typography is used to denote matrices and vectors. Let $\mathbf{I}$ denote the identity matrix with size clear from context. For a collection of points $\{\alpha_{\ell}\}_{0\leq\ell<n}\subset\mathbb{C}$, let $\operatorname{diag}\left\{\left(\alpha_{\ell}\right)_{0\leq\ell<n}\right\}$ denote the $n\times n$ diagonal matrix with $\ell^{th}$ diagonal entry $\alpha_{\ell}$. Let $\mathbf{1}_{m\times n}$ and $\mathbf{0}_{m\times n}$ respectively denote the $m\times n$ matrix of all 1s and all 0s. Let $\mathbf{B}[j,:]$ and $\mathbf{B}[:,\ell]$ denote the $j^{th}$ row and $\ell^{th}$ column of the matrix $\mathbf{B}$, respectively. For matrices $\mathbf{A},\mathbf{B}$, we denote the by $\mathbf{A}\circ\mathbf{B}$ the Hadamard product $\left(\mathbf{A}\circ\mathbf{B}\right)\left[j,\ell\right]=\mathbf{A}\left[j,\ell\right]\mathbf{B}\left[j,\ell\right]$.

2 Generalized Rayleigh Quotient Estimation

2.1 Rayleigh Quotient for Complex Diagonalizable Matrices

Let $\mathbf{B}\in\mathbb{C}^{n\times n}$. There corresponds to $\mathbf{B}$ a (in general non-symmetric) bilinear form $\left\langle\mathbf{f},\mathbf{g}\right\rangle_{\mathbf{B}}\mapsto\mathbf{f}^{*}\mathbf{B}\mathbf{g},$ defined for $\mathbf{f},\mathbf{g}\in\mathbb{C}^{n\times 1}$. The behavior of this bilinear form can be analyzed in a scale-invariant manner through the Rayleigh quotient. When $\mathbf{B}$ is Hermitian, (1) states that the Rayleigh quotient $\langle\mathbf{f},\mathbf{f}\rangle_{\mathbf{B}}/\|\mathbf{f}\|_{2}^{2}$ is controlled for $\mathbf{f}\neq\mathbf{0}_{n\times 1}$ by the largest and smallest eigenvalues of $\mathbf{B}$. We extend this result to the case when $\mathbf{B}$ has complex eigenvalues.

Theorem 2.1**.**

Let $\mathbf{B}\in\mathbb{C}^{n\times n}$ be decomposed as $\mathbf{B}=\mathbf{U}\operatorname{diag}\left\{\boldsymbol{\lambda}\left(\mathbf{B}\right)\right\}\mathbf{V}^{*}$ where $\mathbf{U}\mathbf{V}^{*}=\mathbf{I}$ and $\boldsymbol{\lambda}\left(\mathbf{B}\right)\subset\mathbb{C}$. Write $\lambda_{\ell}(\mathbf{B})=\left|\lambda_{\ell}(\mathbf{B})\right|e^{i\theta_{\ell}}$ for $\theta_{\ell}\in[0,2\pi)$. Let $\mathbf{f},\mathbf{g}\in\mathbb{C}^{n\times 1}$ be such that there exist $\mathbf{F},\mathbf{G}\in\mathbb{C}^{n\times 1}$ satisfying $\mathbf{F}\circ\mathbf{G}\neq\mathbf{0}_{n\times 1}$ and

[TABLE]

Then

[TABLE]

Proof.

Using the eigendecomposition of $\mathbf{B}$ we have

[TABLE]

Writing each eigenvalue in polar form as $\lambda_{\ell}(\mathbf{B})=\left|\lambda_{\ell}(\mathbf{B})\right|e^{i\theta_{\ell}}$ and applying the admissibility condition (2) yields

[TABLE]

The result follows by algebraic manipulation. ∎

The condition (2) is an admissibility condition on the vectors $\mathbf{f},\mathbf{g}$. In the case where $\mathbf{B}$ is Hermitian, the eigenvalues of $\mathbf{B}$ are real and $\mathbf{V}=\mathbf{U}$ is unitary. If $\mathbf{F}\left[\ell\right]\mathbf{G}\left[\ell\right]\geq 0$ for all $\ell$, then

[TABLE]

since $\mathbf{U}=\mathbf{V}$ is unitary. If $\mathbf{B}$ is Hermitian and moreover $\mathbf{f}=\mathbf{g}$, which implies that $\overline{\mathbf{F}}=\mathbf{G}$, we have

[TABLE]

which recovers $(\ref{eqn:ClassicalRayleigh})$. In the general case, $\mathbf{f},\mathbf{g}$ must interact in a particular way for Theorem 2.1 to hold, as quantified by the admissibility condition (2). A slightly more general result holds, using the singular value decomopsition modulo signings of the singular values:

Theorem 2.2**.**

Let $\mathbf{B}\in\mathbb{C}^{n\times n}$ be decomposed as $\mathbf{B}=\mathbf{U}\operatorname{diag}\left\{\boldsymbol{\sigma}\left(\mathbf{B}\right)\right\}\mathbf{V}^{*}$ where $\mathbf{U}\mathbf{U}^{*}=\mathbf{I}=\mathbf{V}\mathbf{V}^{*}$ and $\boldsymbol{\sigma}\left(\mathbf{B}\right)\subset\mathbb{C}$. Write $\sigma_{\ell}(\mathbf{B})=\left|\sigma_{\ell}(\mathbf{B})\right|e^{i\theta_{\ell}}$ for $\theta_{\ell}\in\left\{0,\pi\right\}$. Let $\mathbf{f},\mathbf{g}\in\mathbb{C}^{n\times 1}$ be such that there exist $\mathbf{F},\mathbf{G}\in\mathbb{C}^{n\times 1}$ satisfying $\mathbf{F}\circ\mathbf{G}\neq\mathbf{0}_{n\times 1}$ and

[TABLE]

Then

[TABLE]

Proof.

Using the Singular value decomopsition modulo signings of the singular values of $\mathbf{B}$ we have

[TABLE]

Writing each singular value in polar form as $\sigma_{\ell}(\mathbf{B})=\left|\sigma_{\ell}(\mathbf{B})\right|e^{i\theta_{\ell}}$ and applying the admissibility condition (3) yields

[TABLE]

The result follows by algebraic manipulation. ∎

2.2 Illustration of Admissible Vectors Via Index Rotations

Admissible vectors may be constructed as follows. Consider the matrix transformation prescribed by performing a rotation to the entry indices of a matrix $\mathbf{A}\in\mathbb{C}^{n\times n}$:

[TABLE]

[TABLE]

where rotation angles are restricted to $\theta\in\left\{0,\frac{\pi}{2},\pi,\frac{3\pi}{2}\right\}$. Together with the transpose, these transformations generate the dihedral group of order $8$. Given a matrix $\mathbf{A}$, we have

[TABLE]

[TABLE]

Furthermore, these transformations preserve unitarity:

Lemma 2.3**.**

Suppose $\mathbf{A}$ is unitary. Then $\forall\>\theta\in\left\{0,\frac{\pi}{2},\pi,\frac{3\pi}{2}\right\}$, $\mathbf{A}^{\text{R}_{\theta}}$ is also unitary.

Proof.

It suffices to prove that $\left(\mathbf{A}^{\text{R}_{\frac{\pi}{2}}}\right)^{*}\left(\mathbf{A}^{\text{R}_{\frac{\pi}{2}}}\right)=\mathbf{I}$. Recall that $\mathbf{A}$ unitary means that

[TABLE]

and so $\mathbf{A}$ has rows and columns with $\ell^{2}$ norm equal to 1. Since the entries of the $i^{th}$ row of $\mathbf{A}^{\text{R}_{\frac{\pi}{2}}}$ correspond to a permutation of the entries of the $(n-1-j)^{th}$ column of $\mathbf{A}$ it follows that $\mathbf{A}^{\text{R}_{\frac{\pi}{2}}}$ also has rows and columns with $\ell^{2}$ norm equal to 1. Furthermore since every column of $\mathbf{A}$ undergoes the same permutation when converted into a row of $\mathbf{A}^{\text{R}_{\frac{\pi}{2}}}$ it follows that $\forall\,0\leq j<\ell<n,$

[TABLE]

thus completing the proof. The other angles follow similarly.

∎

Moreover, index rotations obey a convenient multiplicative identity:

Lemma 2.4**.**

Let $\mathbf{A},\mathbf{B}\in\mathbb{C}^{n\times n}$. Then $\left(\mathbf{A}\mathbf{B}\right)^{\text{R}_{\frac{\pi}{2}}}=\mathbf{B}^{\top}\left(\left(\mathbf{A}^{\top}\right)^{\text{R}_{\frac{3\pi}{2}}}\right)^{\top}$.

Proof.

We prove the case when $n$ is odd; the argument is analogous for $n$ even. Recall that

[TABLE]

Applying the rotation operator,

[TABLE]

In particular, if $\theta=\frac{\pi}{2}$ we have

[TABLE]

Similarly,

[TABLE]

In particular, for $\theta=\frac{3\pi}{2}$ we have

[TABLE]

from which the desired claim follows. ∎

We now establish the existence of a class of admissible vectors.

Theorem 2.5**.**

Let $\mathbf{M}\in\mathbb{R}^{n\times n}$ be Hermitian such that $\mathbf{M}^{\text{R}_{\frac{\pi}{2}}}$ is diagonalizable with

[TABLE]

Write the eigenvalues of $\mathbf{M}^{\text{R}_{\frac{\pi}{2}}}$ in the polar form $\lambda_{\ell}\left(\mathbf{M}^{\text{R}_{\frac{\pi}{2}}}\right)=\left|\lambda_{\ell}\left(\mathbf{M}^{\text{R}_{\frac{\pi}{2}}}\right)\right|e^{i\theta_{\ell}},$ $\ell=0,\dots,n-1$. Then for every $0\leq j<n$ there are admissible vector pairs $\mathbf{f},\mathbf{g}\in\mathbb{C}^{n\times 1}$ for which

[TABLE]

and moreover,

[TABLE]

Proof.

For an arbitrary matrix $\mathbf{B}\in\mathbb{C}^{n\times n}$ the following identity follows from Lemma 2.3

[TABLE]

since for any matrix $\mathbf{B}$ we have

[TABLE]

In particular, for a real Hermitian $\mathbf{M}$,

[TABLE]

For some fixed $j$ let

[TABLE]

Note that there exist a unique pair of vectors $\bar{\mathbf{F}},\mathbf{G}\in\mathbb{C}^{n\times 1}$ such that $\mathbf{f}=\mathbf{V}\bar{\mathbf{F}}\;\text{ and }\;\mathbf{g}=\mathbf{U}\mathbf{G}$. In particular, $\mathbf{G}=(0,0,\dots,0,1,0,\dots,0)^{\top}$, with the 1 in the $j^{th}$ coordinate. So, to show admissibility of the vector pair $\mathbf{f},\mathbf{g}\in\mathbb{C}^{n\times 1}$, it thus suffices to show $\mathbf{F}[j]\mathbf{G}[j]e^{i\theta_{j}}\in\mathbb{R}$. This follows from the fact that

[TABLE]

and also

[TABLE]

Moreover, an application of the variational Rayleigh quotient estimates for $\mathbf{M}$ yields

[TABLE]

thus concluding the proof.

∎

We remark that the matrix $\mathbf{M}^{\text{R}_{\frac{\pi}{2}}}$ is a perHermitian matrix [GL12]. Note that the rotation operator preserves the eigenvalues of a Hermitian matrix, though the eigenvectors change in general.

Corollary 2.6**.**

Let $\mathbf{M}$ be a Hermitian matrix with spectral decomposition $\mathbf{M}=\mathbf{V}^{*}\operatorname{diag}\{\boldsymbol{\lambda}(\mathbf{M})\}\mathbf{V}$, $\mathbf{V}^{*}\mathbf{V}=\mathbf{I}$. Then $\mathbf{M}^{\text{R}_{\frac{\pi}{2}}}=\mathbf{V}^{\top}\operatorname{diag}\left\{\boldsymbol{\lambda}\left(\mathbf{M}\right)\right\}\left(\left(\mathbf{V}^{*\top}\right)^{\text{R}_{\frac{3\pi}{2}}}\right)^{\top}$.

Proof.

Applying Lemma 2.4 to the factorization $\mathbf{V}^{*}\left(\operatorname{diag}\{\boldsymbol{\lambda}(\mathbf{M})\}\mathbf{V}\right)$ yields the desired result. ∎

3 Estimating Independent Set Cardinalities in Directed Graphs

As an application of our method to graph theory, we develop estimates on the size of the largest independent set in certain directed graphs. We will consider the inner product on $n\times n$ matrices $\langle\mathbf{B},\mathbf{M}\rangle=\text{tr}(\mathbf{B}^{*}\mathbf{M})$, which has associated norm

[TABLE]

Definition 3.1**.**

Let $\mathcal{G}$ be a directed, unweighted graph on $n$ vertices. The graph $\mathcal{G}$ has adjacency matrix $\mathbf{B}\in\{0,1\}^{n\times n}$ where $\mathbf{B}\left[i,j\right]=1$ if and only if there is a directed edge from the $i^{th}$ node to the $j^{th}$ node in $\mathcal{G}$. The graph $\mathcal{G}$ is $d$-regular if all row and columns sums are equal to $d$. The graph $\mathcal{G}$ is undirected if it has symmetric adjacency matrix.

The Delsarte-Hoffman bound [Del73, Hof03] is a classical estimate on the cardinality of the largest independent set of an undirected, unweighted, $d$-regular graph in terms of the spectrum of its adjacency matrix. We state it and provide a proof for completeness.

Theorem 3.2**.**

(Undirected Delsarte-Hoffman Bound). Let $\mathbf{B}\in\{0,1\}^{n\times n}$ be the adjacency matrix of an undirected $d$-regular graph $\mathcal{G}$ with spectral decomposition

[TABLE]

where $\boldsymbol{\lambda}(\mathbf{B})\subset\mathbb{R}$. Let $I$ be the indices of an independent set in $\mathcal{G}$. Then

[TABLE]

Proof.

Note that by $d$-regularity, $\mathbf{B}$ has an eigenvalue of $d$; without loss of generality, let $\lambda_{0}(\mathbf{B})=d$. Then $\mathbf{B}=\frac{d}{n}\mathbf{1}_{n\times n}+\left(\mathbf{I}-\frac{\mathbf{1}_{n\times n}}{n}\right)\mathbf{B}.$ Thus, for all $\mathbf{f}\in\mathbb{C}^{n\times 1}$,

[TABLE]

Let $\mathbf{B}=\mathbf{U}\operatorname{diag}\{\mathbb{\lambda}(\mathbf{B})\}\mathbf{U}^{*}$ where $\mathbf{U}\mathbf{U}^{*}=\mathbf{I}$. We analyze the second summand on the right-hand side as follows:

[TABLE]

Hence,

[TABLE]

To every independent set $I$, there is a corresponding indicator vector $\mathbf{f}=\mathbf{1}_{I}$ for which by definition $\langle\mathbf{B},\mathbf{f}\,\mathbf{f}^{*}\rangle=\mathbf{f}^{*}\mathbf{B}\mathbf{f}=0$. For such an indicator vector $\mathbf{f}$, it follows that

[TABLE]

Noting that $\left|I\right|=\|\mathbf{f}\|_{2}^{2}$, we get

[TABLE]

[TABLE]

thus completing the proof. ∎

The condition on the maximum size on an independent set may be characterized as the maximum value $\|\mathbf{1}_{I}\|_{2}^{2}$, where $\mathbf{1}_{I}^{*}\mathbf{B}\mathbf{1}_{I}=0$ for some $I\subset V$. The spectral decomposition of $\mathbf{B}$ decouples the rank-one matrix $\mathbf{1}_{n\times n}$ associated with the eigenvalue $\lambda_{0}(\mathbf{B})=d$, from whence the analysis flows. Note that the Delsarte-Hoffman bound is sharp for the complete bipartite graph having $n$ vertices in each partition, since in this case

[TABLE]

3.1 Directed Delsarte-Hoffman Bound

We now consider independent sets in directed regular graphs and broaden the scope to adjacency matrices whose entries are not necessarily binary.

Definition 3.3**.**

Let $\mathcal{G}$ be a directed graph with $n$ nodes. A matrix $\mathbf{B}\in\mathbb{C}^{n\times n}$ is a pseudo-adjacency matrix for $\mathcal{G}$ if $\mathbf{B}_{ij}=0$ whenever there is no directed edge from the $i^{th}$ node to the $j^{th}$ node in $\mathcal{G}$. The pseudo-adjacency matrix $\mathbf{B}$ is said to be $\delta$-regular if all row and column sums of $\mathbf{B}$ are equal to $\delta$.

We develop Delsarte-Hoffman-type bounds based on the spectral decomposition and the singular value decomposition of the (non-Hermitian) pseudo-adjacency matrix.

Theorem 3.4**.**

Let $\mathbf{B}\in\mathbb{C}^{n\times n}$ denote a $\delta$-regular pseudo-adjacency matrix for a graph $\mathcal{G}$. Let $\mathbf{B}$ be decomposed as $\mathbf{B}=\mathbf{U}\operatorname{diag}\left\{\boldsymbol{\lambda}\left(\mathbf{B}\right)\right\}\mathbf{V}^{*}$ where $\mathbf{U}\mathbf{V}^{*}=\mathbf{I}$. Let $\lambda_{\ell}(\mathbf{B})=\alpha_{\ell}e^{i\theta_{\ell}}$, $\alpha_{\ell}\in\mathbb{R},\theta_{\ell}\in[0,2\pi)$ be a polar form of the $\ell$-th eigenvalue of $\mathbf{B}$. Let $\boldsymbol{\alpha}(\mathbf{B})=(\alpha_{0},\dots,\alpha_{n-1})$. Let $\mathbf{f},\mathbf{g}\in\{0,1\}^{n\times 1}$ be such that there exist $\mathbf{F},\mathbf{G}\in\mathbb{C}^{n\times 1}$ satisfying

[TABLE]

Then if $\mathbf{f}$ and $\mathbf{g}$ denote respectively indicator vectors for rows and columns associated with a rectangular 0 block in $\mathbf{B}$,

[TABLE]

Proof.

Note that by $\delta$-regularity, $\mathbf{B}$ has an eigenvalue of $\delta$; without loss of generality, let $\lambda_{0}(\mathbf{B})=\delta$. Then we have that $\mathbf{U}[:,0]=\mathbf{V}[:,0]=\frac{1}{\sqrt{n}}(1,1,\dots,1)^{\top}$. Hence, $\mathbf{B}=\frac{\delta}{n}\mathbf{1}_{n\times n}+\left(\mathbf{I}-\frac{\mathbf{1}_{n\times n}}{n}\right)\mathbf{B}.$ Thus, for all $\mathbf{f},\mathbf{g}\in\mathbb{C}^{n\times 1}$ subject to (6),

[TABLE]

We analyze the second term of the right hand side as follows:

[TABLE]

Note that $\mathbf{F}\left[0\right]=\frac{1}{\sqrt{n}}\|\mathbf{f}\|_{1}=\frac{1}{\sqrt{n}}\|\mathbf{f}\|_{2}^{2}$ follows from the observation that $\mathbf{f}$ takes values in $\{0,1\},\mathbf{U}^{*}\mathbf{f}=\mathbf{U}^{*}\mathbf{V}\overline{\mathbf{F}}=\overline{\mathbf{F}}$, and $\mathbf{U}[:,0]=\frac{1}{\sqrt{n}}(1,\dots,1)^{\top}$. That $\mathbf{G}\left[0\right]=\frac{1}{\sqrt{n}}\|\mathbf{g}\|_{2}^{2}$ follows similarly. Hence,

[TABLE]

Since $\mathbf{f}$ and $\mathbf{g}$ are indicator vectors for rows and columns associated with a rectangular 0 block in $\mathbf{B}$, $\mathbf{f}^{*}\mathbf{B}\mathbf{g}=0$. It follows that

[TABLE]

from whence the result follows by algebraic manipulation.

∎

In the case that $\mathbf{B}$ is the adjacency matrix of a $d$-regular graph, the result may be interpreted as a generalization of Theorem 3.2.

Corollary 3.5**.**

(Directed Delsarte-Hoffman Bound) Let $\mathbf{B}\in\{0,1\}^{n\times n}$ denote a $d$-regular adjacency matrix for a graph $\mathcal{G}$ on $n$ vertices. Let $\mathbf{B}$ be decomposed as $\mathbf{B}=\mathbf{U}\operatorname{diag}\left\{\boldsymbol{\lambda}\left(\mathbf{B}\right)\right\}\mathbf{V}^{*}$ where $\mathbf{U}\mathbf{V}^{*}=\mathbf{I}$. Let $\lambda_{\ell}(\mathbf{B})=\alpha_{\ell}e^{i\theta_{\ell}},$ $\alpha_{\ell}\in\mathbb{R},\theta_{\ell}\in[0,2\pi)$ be a polar decomposition of $\boldsymbol{\lambda}(\mathbf{B})$. Let $\boldsymbol{\alpha}(\mathbf{B})=(\alpha_{1},\dots,\alpha_{n})$. Let $\mathbf{f},\mathbf{g}\in\{0,1\}^{n\times 1}$ be such that there exist $\mathbf{F},\mathbf{G}\in\mathbb{C}^{n\times 1}$ satisfying

[TABLE]

Then if $\mathbf{f}$ and $\mathbf{g}$ denote respectively indicator vectors for rows and columns associated with a rectangular 0 block in $\mathbf{B}$,

[TABLE]

Note that the quantity bounded in Corollary 3.5 may be interpreted as the geometric average of the size of independent set with respect to “in” and “out” nodes. Indeed, if $\mathbf{f}=\mathbf{1}_{I_{\text{row}}},\mathbf{g}=\mathbf{1}_{I_{\text{col}}}$ are the indicator functions for the in (row) and out (column) vertices of a directed independent set, then $\|\mathbf{f}\|_{2}\|\mathbf{g}\|_{2}=\sqrt{\left|I_{\text{row}}\right|\left|I_{\text{col}}\right|}$.

If $\mathbf{B}$ is Hermitian and $\mathbf{f}=\mathbf{g}$, the admissibility condition (6) is always satisfied as $\theta_{\ell}=0,\ell=0,\dots,n-1$. Indeed, in this case,

[TABLE]

so that the conclusion of Theorem 3.2 holds. Hence, Corollary 3.5 is a strict generalization of the classical Delsarte-Hoffman inequality.

3.1.1 Tightness of Directed Delsarte-Hoffman Bound

When $n$ is a multiple of 4, adjacency matrices of the form

[TABLE]

show Corollary 3.5 is tight. Indeed, when $n=4$, this is a 1-regular directed graph with non-Hermitian adjacency matrix $\mathbf{B}$, which may be decomposed as:

[TABLE]

In this case, the largest independent set has size 2, corresponding to the zero block on the upper left and lower right of the matrix. Let $\mathbf{f}=\mathbf{g}=(1,1,0,0)^{\top}$, so that $\mathbf{F}=\mathbf{G}=\left(-1,0,0,-1\right)^{\top}$. Decomposing the first and fourth eigenvalues as $\alpha_{0}=-1,\alpha_{3}=1,\theta_{0}=\theta_{3}=0$, it is seen that the admissibility condition is satisfied, and that $\min\{\boldsymbol{\alpha}\}=-1$. Moreover,

[TABLE]

so that the estimate of Corollary 3.5 is

[TABLE]

which is tight since $\|\mathbf{f}\|_{2}\|\mathbf{g}\|_{2}=2$. A similar argument holds for the block corresponding to indicator functions $\mathbf{f}=\mathbf{g}=(0,0,1,1)^{\top}$. Together, this shows the maximal independent set of this directed graph is tightly estimated by Corollary 3.5.

3.2 A Delsarte-Hoffman Bound Using the Singular Value Decomposition

Consider the singular value decomposition of $\mathbf{B}\in\mathbb{R}^{n\times n}$ expressed by

[TABLE]

where each element of $\boldsymbol{\sigma}(\mathbf{B})$ is positive. Theorem 3.6 provides a Delsarte-Hoffman estimate on the size of the independent set using the SVD, which holds for all matrices, not just diagonalizable ones.

Theorem 3.6**.**

Let $\mathbf{B}\in\mathbb{C}^{n\times n}$ be a $\delta$-regular pseudo-adjacency matrix of a directed graph. Suppose $\mathbf{B}$ has a decomposition $\mathbf{B}=\mathbf{U}\operatorname{diag}\left(\boldsymbol{\sigma}\right)\mathbf{V}^{*}$ such that $\mathbf{U}\mathbf{U}^{*}=\mathbf{I}=\mathbf{V}\mathbf{V}^{*}$ and $\boldsymbol{\sigma}=(\sigma_{1},\dots,\sigma_{n})$. Let $\sigma_{\text{min}}=\min_{\ell}\sigma_{\ell}$. Suppose that $\mathbf{f},\mathbf{g}\in\{0,1\}^{n\times 1}$ correspond to the indicator sets for row and column indices respectively of a rectangular zero block, and that there exist $\mathbf{F},\mathbf{G}\in\mathbb{C}^{n\times 1}$ such that

[TABLE]

Then

[TABLE]

Proof.

By the SVD and by $\delta$-regularity,

[TABLE]

Analyzing the second term for all $\mathbf{f},\mathbf{g}\in\mathbb{C}^{n\times 1}$, subject to (8),

[TABLE]

Note that $\mathbf{F}\left[0\right]=\frac{1}{\sqrt{n}}\|\mathbf{f}\|_{1}=\frac{1}{\sqrt{n}}\|\mathbf{f}\|_{2}^{2}$ follows from $\mathbf{U}^{*}\mathbf{f}=\mathbf{U}^{*}\mathbf{U}\overline{\mathbf{F}}=\mathbf{F}$ and that fact that $\mathbf{f}\in\{0,1\}^{n\times 1}$, $\mathbf{U}[:,0]=\frac{1}{\sqrt{n}}(1,\dots,1)^{\top}$; $\mathbf{G}\left[0\right]=\frac{1}{\sqrt{n}}\|\mathbf{g}\|_{2}^{2}$ follows similarly. Thus,

[TABLE]

Every zero block $I$ is specified by a pair of indicator vectors $\mathbf{f}=\mathbf{1}_{I_{L}}$ and $\mathbf{g}=\mathbf{1}_{I_{R}}$ such that $\mathbf{f}^{*}\mathbf{B}\mathbf{g}=0$. For such an $\mathbf{f}$, $\mathbf{g}$ pair also subject to the admissibility condition (8) we have

[TABLE]

from whence the result follows by algebraic manipulation.

∎

Theorem 3.6 requires a decomposition which bears resemblance to the SVD in the fact that $\mathbf{B}=\mathbf{U}\operatorname{diag}(\boldsymbol{\sigma})\mathbf{V}^{*}$, where $\mathbf{U}\mathbf{U}^{*}=\mathbf{I}=\mathbf{V}\mathbf{V}^{*}$, but without the condition that $\sigma_{\ell}\geq 0$ for all $\ell$. Note that if $\sigma_{\ell}\mapsto-\sigma_{\ell}$, and $\mathbf{U}[:,\ell]\mapsto-\mathbf{U}[:,\ell]$ or $\mathbf{V}^{*}[\ell,:]\mapsto-\mathbf{V}^{*}[\ell,:]$, this still expresses such a decomposition for $\mathbf{B}$. In this sense, there are $2^{n}$ decompositions to consider in Theorem 3.6, corresponding to the $2^{n}$ possible sign assignments. Thus, one can think of the decomposition in Theorem 3.6 as a (non-unique) signed SVD, and the condition (8) as an admissibility condition with respect to this decomposition.

If in particular $\mathbf{B}$ is the adjacency matrix of a $d$-regular graph, the following result holds.

Corollary 3.7**.**

Let $\mathbf{B}\in\{0,1\}^{n\times n}$ be a $d$-regular adjacency matrix of a directed graph. Suppose $\mathbf{B}$ has a decomposition $\mathbf{B}=\mathbf{U}\operatorname{diag}\left(\boldsymbol{\sigma}\right)\mathbf{V}^{*}$ such that $\mathbf{U}\mathbf{U}^{*}=\mathbf{I}=\mathbf{V}\mathbf{V}^{*}$ and $\boldsymbol{\sigma}=(\sigma_{1},\dots,\sigma_{n})$. Let $\sigma_{\text{min}}=\min_{\ell}\sigma_{\ell}$. Suppose that $\mathbf{f}\in\{0,1\}^{n\times 1},\mathbf{g}\in\{0,1\}^{n\times 1}$ correspond to the indicator sets for row and column indices respectively of a rectangular zero block, and that there exist $\mathbf{F},\mathbf{G}\in\mathbb{C}^{n\times 1}$ such that

[TABLE]

Then

[TABLE]

4 Discussion and Future Research

This article proposes generalizations of classical linear algebraic and spectral graph theoretic results to the case in which the underlying matrix $\mathbf{B}$ is non-Hermitian. This is done by constraining certain vectors to satisfy admissibility conditions. When $\mathbf{B}$ is Hermitian, these admissibility conditions hold and the classical results are recovered. The admissibility condition take slightly different forms, depending on which decomposition is used in place of the spectral decomposition into an orthonormal eigenbasis.

In Theorems 2.1, 3.4, $\mathbf{B}$ is assumed diagonalizable as $\mathbf{B}=\mathbf{U}\operatorname{diag}(\boldsymbol{\lambda}(\mathbf{B}))\mathbf{V}^{*}$ where $\boldsymbol{\lambda}(\mathbf{B})$ may be complex and $\mathbf{U},\mathbf{V}$ need not be unitary, merely inverses: $\mathbf{U}\mathbf{V}^{*}=\mathbf{I}=\mathbf{V}^{*}\mathbf{U}$. The analysis of $\mathbf{f}^{*}\mathbf{B}\mathbf{g}$ proceeds by assuming $\mathbf{f}$ admits an expansion in terms of the rows of $\mathbf{V}$ and $\mathbf{g}$ an expansion in terms of the rows of $\mathbf{U}$. Of course, when $\mathbf{U}=\mathbf{V}$ these conditions are the same, and when $\mathbf{f}=\mathbf{g}$, this condition always holds. On the other hand, Theorem 3.6 takes advantage of the singular value decomposition $\mathbf{B}=\mathbf{U}\operatorname{diag}(\boldsymbol{\sigma}(\mathbf{B}))\mathbf{V}^{*}$ where $\mathbf{U}$ and $\mathbf{V}$ are unitary but $\mathbf{U}\neq\mathbf{V}$. The analysis of $\mathbf{B}$ in this situation requires a different condition on $\mathbf{f},\mathbf{g}$, namely that $\mathbf{f}$ has an admissible decomposition with respect to the rows of $\mathbf{U}$, and $\mathbf{g}$ with respect to the rows of $\mathbf{V}$. We remark that in all of these cases, the crucial property is that for $d$-regular unweighted graphs (or $\delta$-regular weighted graphs), the first eigenvector or singular vector (both left and right) is the vector $\frac{1}{\sqrt{n}}(1,1,\dots,1)^{\top}\in\mathbb{R}^{n\times 1}$ with corresponding eigenvalue or singular value $d$. All subsequent analysis is downstream from this observation.

Intuitively, as $\mathbf{B}$ deviates from being Hermitian, the admissibility conditions will still hold for a large class of vectors $\mathbf{f},\mathbf{g}$. A topic of future research is to develop a rigorous perturbation theory of Hermitian matrices that quantifies how likely the admissibility conditions are to hold in a probabilistic sense. That is, if $\mathbf{B}$ is Hermitian, then the admissibility condition holds automatically for all $\mathbf{f}=\mathbf{g}$. As $\mathbf{f}$ deviates from $\mathbf{g}$ and $\mathbf{B}$ deviates from Hermiticity, it is of interest to determine which vectors (or, what proportion of them in a probabilistic sense) satisfy the admissibility condition.

5 Acknowledgements

We are grateful to Jim Fill (Johns Hopkins University), Yuval Filmus (Technion), and Xiaoqin Guo (University of Wisconsin, Madison) for insightful comments regarding the results and presentation of this manuscript.