On the Relationship Between Real and Complex Linear Systems

TL;DR

This paper explores the conditions under which complex linear systems involving conjugates can be simplified to standard complex linear systems, revealing new links between real and complex systems with practical numerical methods.

Contribution

It characterizes when such systems reduce to complex linear systems and demonstrates how to construct these systems, offering new insights into the connection between real and complex linear algebra.

Findings

Real symmetric systems can be solved via complex linear systems.

Provides a method to construct complex systems from conjugate-involving systems.

Numerical illustrations validate the theoretical results.

Abstract

We consider the problem of solving a linear system of equations which involves complex variables and their conjugates. We characterize when it reduces to a complex linear system, that is, a system involving only complex variables (and not their conjugates). In that case, we show how to construct the complex linear system. Interestingly, this provides a new insight on the relationship between real and complex linear systems. In particular, any real symmetric linear system of equations can be solved via a complex linear system of equations. Numerical illustrations are provided. The mathematics in this manuscript constitute an exciting interplay between Schur's complement, Cholesky's factorization, and Cauchy's interlace theorem.

Equations

$$Mz+N\bar{z}=p,z\in {\mathbb{C}}^n.$$

$$Az=b,z\in {\mathbb{C}}^n$$

$$\mathcal{R}(M,N):=\{p\in {\mathbb{C}}^n|\exists z\in {\mathbb{C}}^n:Mz+N\bar{z}=p\}.$$

$${\mathbb{C}}^{2n}=\mathcal{R}\left(\begin{array}{c}N\\ \overline{M}\end{array}\right)\oplus {\left[\mathcal{R}\left(\begin{array}{c}N\\ \overline{M}\end{array}\right)\right]}^{\perp }$$

$$\begin{array}[]{c}\{z\in\mathbb{C}^{n}~{}|~{}Mz+N\bar{z}=p\}\\[5.69046pt] =\\[5.69046pt] \{z\in\mathbb{C}^{n}~{}~{}|~{}~{}(UM+V\overline{N})z=Up+V\overline{p}~{}\}.\end{array}$$

$$P_{\perp }\left(\begin{array}{c}z\\ \bar{z}\end{array}\right)=0.$$

$$\left(\begin{array}{c}z\\ \bar{z}\end{array}\right)\in \mathcal{R}\left(\begin{array}{c}N\\ \overline{M}\end{array}\right).$$

$$\left(\begin{array}{c}z\\ \bar{z}\end{array}\right)=\left(\begin{array}{c}Nw\\ \overline{M}w\end{array}\right)$$

$$Nw-M\overline{w}=0.$$

$$Mz+N\bar{z}=p$$

$$\left(\begin{array}{c}Mz\\ \overline{N}z\end{array}\right)+\left(\begin{array}{c}N\bar{z}\\ \overline{M}\bar{z}\end{array}\right)=\left(\begin{array}{c}p\\ \overline{p}\end{array}\right)$$

$$P_{\perp }\left(\begin{array}{c}Mz\\ \overline{N}z\end{array}\right)+P_{\perp }\left(\begin{array}{c}N\bar{z}\\ \overline{M}\bar{z}\end{array}\right)=P_{\perp }\left(\begin{array}{c}p\\ \overline{p}\end{array}\right)$$

$$P_{\perp }\left(\begin{array}{c}Mz\\ \overline{N}z\end{array}\right)=P_{\perp }\left(\begin{array}{c}p\\ \overline{p}\end{array}\right)$$

$$P_{\perp }\left(\begin{array}{c}N\bar{z}\\ \overline{M}\bar{z}\end{array}\right)=0.$$

$$(\hat{U}M+\hat{V}\overline{N})z=\hat{U}p+\hat{V}\overline{p}.$$

$$\begin{array}[]{c}\forall i\in I,~{}\forall j\in\{1,\ldots,n\},~{}U_{ij}:=\hat{U}_{ij},~{}V_{ij}:=\hat{V}_{ij}.\end{array}$$

$$P_{\perp }=\left(\begin{array}{cc}{P}_{11}& P_{12}\\ P_{21}& P_{22}\end{array}\right)$$

$$P_{\perp }\left(\begin{array}{c}z\\ \bar{z}\end{array}\right)=\left(\begin{array}{cc}{P}_{11}& P_{12}\\ P_{21}& P_{22}\end{array}\right)\left(\begin{array}{c}x+iy\\ x-iy\end{array}\right)=\left(\begin{array}{cc}{P}_{11}+P_{12}& i(P_{11}-P_{12})\\ P_{21}+P_{22}& i(P_{21}-P_{22})\end{array}\right)\left(\begin{array}{c}x\\ y\end{array}\right)$$

$$\text{Ker}\left(\begin{array}{cc}\Re P_{11}+\Re P_{12}& \Im P_{12}-\Im P_{11}\\ \Re P_{21}+\Re P_{22}& \Im P_{22}-\Im P_{21}\\ \Im P_{11}+\Im P_{12}& \Re P_{11}-\Re P_{12}\\ \Im P_{21}+\Im P_{22}& \Re P_{21}-\Re P_{22}\end{array}\right)=\{0\}$$

$$M:=\left(\begin{array}{ccccc}0& -i& 0& 2-i& 5i\\ 0& 3i& 0& 3& 9i\\ -1& 5& 1-3i& -3+3i& 1+7i\\ 0& -2i& 0& -i& -i\\ 0& 1-i& 0& i& -2-3i\end{array}\right),$$

$$N:=\left(\begin{array}{ccccc}-i& 5i& -3+i& 3-3i& 7+i\\ 0& -3+2i& 0& -2& -1+3i\\ 0& -1& 0& -1+2i& 5\\ 0& 1& 0& i& -i\\ 0& 5& 0& 7+4i& 1+2i\end{array}\right),$$

$$p:=\left(\begin{array}{c}1-i\\ 3\\ -1+i\\ 5+i\\ 1\end{array}\right).$$

$$P_{\perp }\left(\begin{array}{c}M\\ \overline{N}\end{array}\right).$$

$$U:=\left(\begin{array}{ccccc}0.0000+0.0000i& -0.1686-0.0351i& 0.6782+0.0000i& 0.0612+0.0175i& -0.0133-0.0054i\\ 0.0000+0.0000i& 0.1592-0.0009i& 0.0612-0.0175i& 0.9171+0.0000i& 0.0044-0.1532i\\ 0.0000+0.0000i& 0.0309-0.0069i& -0.0133+0.0054i& 0.0044+0.1532i& 0.0329+0.0000i\\ 0.0000+0.0000i& -0.0351+0.1686i& 0.0000+0.3218i& 0.0175-0.0612i& -0.0054+0.0133i\\ 0.0000+0.0000i& 0.2393-0.1929i& -0.0152+0.1880i& -0.0881+0.0651i& 0.0169+0.0212i\end{array}\right)$$

$$V:=\left(\begin{array}{ccccc}0.0000-0.3218i& -0.0152-0.1880i& 0.0000+0.0000i& -0.0128-0.1089i& 0.1766+0.0448i\\ 0.0175+0.0612i& -0.0881-0.0651i& 0.0000+0.0000i& 0.0510+0.0511i& 0.0000+0.0433i\\ -0.0054-0.0133i& 0.0169-0.0212i& 0.0000+0.0000i& -0.0523+0.0314i& -0.0300-0.0397i\\ 0.6782+0.0000i& -0.1880+0.0152i& 0.0000+0.0000i& -0.1089+0.0128i& 0.0448-0.1766i\\ -0.1880-0.0152i& 0.4239+0.0000i& 0.0000+0.0000i& 0.1337-0.0184i& 0.2104-0.0583i\end{array}\right).$$

$$A:=\left(\begin{array}{ccccc}-0.3563+0.0000i& 2.4429+0.0891i& 0.3563-1.0690i& -0.2151+0.8851i& 0.5264+0.8788i\\ -0.1225+0.0349i& 0.6516-1.0495i& 0.0178-0.4024i& 0.7686-0.0611i& -0.1645+2.0554i\\ 0.0266-0.0107i& -0.0681-0.0324i& 0.0057+0.0905i& -0.0774-0.1673i& -0.1734-0.0619i\\ 0.0000+0.3563i& 0.0891-2.4429i& -1.0690-0.3563i& 0.8851+0.2151i& 0.8788-0.5264i\\ 0.0303-0.3759i& 0.5094+1.6202i& 1.0975+0.4669i& 0.0983-3.0761i& -1.1428+0.6953i\end{array}\right)$$

$$b:=\left(\begin{array}{c}-0.6283-0.6565i\\ 5.0216+0.9709i\\ -0.1995+0.8183i\\ -0.6565+0.6283i\\ 2.0157-1.0104i\end{array}\right).$$

$$\left(\begin{array}{c}-27.4310+50.9483i\\ -4.0647+5.7543i\\ 0.0000+0.0000i\\ 2.7694-1.2220i\\ 0.6875+0.9203i\end{array}\right)+\mathbb{C}\left(\begin{array}{c}1-3i\\ 0\\ 1\\ 0\\ 0\end{array}\right)$$

$$\begin{array}[]{c}\forall p\in\mathbb{C}^{n},~{}\exists!z\in\mathbb{C}^{n}:~{}Mz+N\bar{z}=p\\[5.69046pt] \Longleftrightarrow\\[5.69046pt] \begin{pmatrix}M&N\\[2.84544pt] \overline{N}&\overline{M}\end{pmatrix}~{}\text{is invertible.}\end{array}$$

$$\begin{array}[]{c}\{z\in\mathbb{C}^{n}~{}|~{}Mz+N\bar{z}=p\}\\[5.69046pt] =\\[5.69046pt] \{z\in\mathbb{C}^{n}~{}~{}|~{}~{}(UM+V\overline{N})z=Up+V\overline{p}~{}\}\end{array}$$

Full text

\slugger

mmsxxxxxxxx–x

1 Introduction

Let $\mathbb{N}$, $\mathbb{R}$ and $\mathbb{C}$ respectively denote the set of natural, real and complex numbers, and let $i$ denote the imaginary number satisfying $i^{2}=-1$. Let $\Re z$, $\Im z$, and $\bar{z}$ respectively denote the real part, the imaginary part, and the complex conjugate of $z\in\mathbb{C}$. Let $\mathcal{M}_{n}(\mathbb{R})$ and $\mathcal{M}_{n}(\mathbb{C})$ respectively denote the set of real-valued and complex-valued square matrices of size $n\in\mathbb{N}$. Consider $M,N\in\mathcal{M}_{n}(\mathbb{C})$ and $p\in\mathbb{C}^{n}$ and the system

[TABLE]

The set of solutions to (1) is a real affine space, but not a complex affine space in general. For instance, if $n=1$, $M=N=1$, and $p=0$, then the system reads $z+\bar{z}=0,~{}z\in\mathbb{C}$. We raise the question of when system (1) can be reduced to a complex linear system

[TABLE]

where $A\in\mathcal{M}_{n}(\mathbb{C})$ and $b\in\mathbb{C}^{n}$, and if so, how $A$ and $b$ can be constructed. Naturally, the set of solutions to (2) is a complex affine space. Thus the system reduction can take place if and only if the set of solutions of system (1) is a complex affine space.

The paper is organized as follows. Section 2 contains the main results which are applied to real linear systems in Section 3. Section 4 concludes our work.

2 Reduction to complex linear system

When system (1) has no solution, it can be reduced to a complex linear system (2) with $A=0$ and $b=p$. When it has one solution or more, Proposition 1 provides a constructive reduction. We thus introduce the notation $\mathcal{R}(M,N)$ to denote the range of system (1), that is to say

[TABLE]

Let’s equip $\mathbb{C}^{2n}$ with Hermitian inner product $\langle\cdot,\cdot\rangle$ defined by $\langle u,v\rangle=\sum\limits_{k=1}^{2n}u_{k}\overline{v}_{k}$. Let $\mathcal{R}$ and Ker respectively denote the range and Kernel of a linear application. The key ingredient to obtain the reduction is the direct sum

[TABLE]

and, in particular, the orthogonal projection $P_{\perp}$ onto the latter set in the direct sum.

Proposition 1**.**

Let $n\in\mathbb{N}$ and $M,N\in\mathcal{M}_{n}(\mathbb{C})$. The following are equivalent:

$\{z\in\mathbb{C}^{n}~{}|~{}Mz+N\bar{z}=0\}$* is a complex vector space.* 2. 2.

The application defined from $\mathbb{C}^{n}$ to $\mathbb{C}^{2n}$ defined by $z\longmapsto P_{\perp}\begin{pmatrix}z\\ \bar{z}\end{pmatrix}$ is injective. 3. 3.

$\exists U,V\in\mathcal{M}_{n}(\mathbb{C}):~{}\forall p\in\mathcal{R}(M,N),$**

[TABLE]

Proof.

The implication $(3\Longrightarrow 1)$ follows from taking $p=0$. As for $(1\Longrightarrow 2)$, consider $z\in\mathbb{C}^{n}$ such that

[TABLE]

Thus

[TABLE]

As a consequence, there exists $w\in\mathbb{C}^{n}$ such that

[TABLE]

The first line minus the conjugate of the second line yields

[TABLE]

Multiplying by the imaginary number, we get $M(\overline{iw})+N(iw)=0$. Thus $\overline{iw}$ belongs to $\{z\in\mathbb{C}^{n}~{}|~{}Mz+N\bar{z}=0\}$, which is a complex vector space thanks to Point 1. Thus $i(\overline{iw})=\overline{w}$ also belongs to it, so that $M\overline{w}+Nw=0$. Together with (9), this implies that $Nw=M\overline{w}=0$. Going back to (8), it holds that $z=0$.

We now prove $(2\Longrightarrow 3)$. Let $p\in\mathcal{R}(M,N)$ and consider $z\in\mathbb{C}^{n}$. Point 2 implies that the following four equations are equivalent:

[TABLE]

[TABLE]

[TABLE]

[TABLE]

since, by definition of $P_{\perp}$, we have

[TABLE]

Let $\begin{pmatrix}\hat{U}~{}\hat{V}\end{pmatrix}$ denote the matrix representation of $P_{\perp}$ in the canonical basis of $\mathbb{C}^{2n}$. The complex matrices $\hat{U}$ and $\hat{V}$ have $2n$ rows and $n$ columns. Equation (13) is thus equivalent to

[TABLE]

The matrix $\hat{U}M+\hat{V}\overline{N}$ has $2n$ rows and $n$ columns so the above system has $2n$ equations and $n$ unknowns. We would like to obtain a system with $n$ equations. First, note that it has at least one solution because $p\in\mathcal{R}(M,N)$. Second, a linear system of equations with more equations than unknowns either has no solutions, or any equation that is a linear combination of other equations can be removed without changing the set of solutions. Thus, consider a set $I\subset\{1,\ldots,2n\}$ of $n$ rows that generate the row space of $\hat{U}M+\hat{V}\overline{N}$. Define $U,V\in\mathcal{M}_{n}(\mathbb{C})$ such that

[TABLE]

We stress that $U$ and $V$ do not depend on the vector $p$, but only on the matrices $M$ and $N$. We now decude that (15) is equivalent to $(UM+V\overline{N})z=Up+V\overline{p}$, which terminates the proof. ∎

Point 2 in Proposition 1 provides a necessary and sufficient condition on $M$ and $N$ to determine whether system (1) can be reduced to a system of the form (2). Let’s write the condition more explicitly. The projection $P_{\perp}$ is an endormorphism of $\mathbb{C}^{2n}$, so let’s consider its block decomposition

[TABLE]

where $P_{11},P_{12},P_{11},P_{11}$ are endormorphisms of $\mathbb{C}^{n}$. For all $z\in\mathbb{C}^{n}$, we then have

[TABLE]

where $x:=\Re z$ and $y:=\Im z$. Point 2 is thus equivalent to

[TABLE]

where the domain of the application in (19) is $\mathbb{C}^{4n}$ and its co-domain is $\mathbb{C}^{2n}$.

We illustrate Proposition 1 with the following example:

[TABLE]

[TABLE]

[TABLE]

Using a reduced column echelon form and a Grahm-Schmidt orthogonalization, we compute the matrix associated to $P_{\perp}$ in the canonical basis of $\mathbb{C}^{2n}$. We then check whether the system can be reduced using (19). In this example, reduction is possible. We then compute a set of 5 rows $I=\{3,4,5,6,7\}$ that generate the row space of

[TABLE]

We then deduce

[TABLE]

and

[TABLE]

Next, we compute the reduced complex linear system (2) where

[TABLE]

and

[TABLE]

We can now compute the solutions to the reduced system:

[TABLE]

We then arbitrarily chose one solution to the reduced system, and then check whether it satisfies the original system. If so, then the set of solutions to the original system and to the reduced system coincide. If not, then the original system has no solutions. In this example, the original system and the reduced system coincide.

We now turn our attention to the special case when the system (1) has a unique solution. When in addition $M$ or $N$ is invertible, there exists an analytic expression for the matrices $U$ and $V$ in Proposition 1 that define the reduction to a complex linear system. Below, the matrix $I$ denotes the identity of $\mathcal{M}_{n}(\mathbb{C})$.

Proposition 2**.**

Let $n\in\mathbb{N}$ and $M,N\in\mathcal{M}_{n}(\mathbb{C})$.

[TABLE]

Assume that either equivalent property holds in (25). Then, for all $p\in\mathbb{C}^{n}$,

[TABLE]

where $U:=I$ and $V:=-N\overline{M}^{-1}$ if $M$ is invertible, and where $U:=N^{-1}$ and $V:=-\overline{M}^{-1}$ if $M$ and $N$ are invertible.

Proof.

($\Longrightarrow$) Consider $z,w\in\mathbb{C}^{n}$ such that

[TABLE]

This implies after conjugation that

[TABLE]

Adding the two previous equations yields

[TABLE]

Thus $z+w$ is a solution to system (1) for $p=0$, whose unique solution is [math]. Hence $z+w=0$. As a result, $Mz-N\bar{z}=0$, that is to say $M(iz)+N(\overline{iz})=0$. Again by unicity of the solution, $iz=0$. To conclude, $z=w=0$.

($\Longleftarrow$) Existence of a solution: there exists $z,w\in\mathbb{C}^{n}$ such that

[TABLE]

which implies after conjugation that

[TABLE]

Since the system has a unique solution, it must be that $z=w$. Thus $z$ is a solution to (1).

Unicity of the solution: consider $z,w\in\mathbb{C}^{n}$ such that $Mz+N\bar{z}=p$ and $Mw+N\overline{w}=p$. It must be that $M(z-w)+N\overline{(z-w)}=0$, thus

[TABLE]

Since the above block matrix is invertible, it follows that $z=w$.

To find suitable choices for the matrices $U$ and $V$, it suffices for them to satisfy that $UN+V\overline{M}=0$ and that the application from $\mathbb{C}^{n}$ to itself defined by

[TABLE]

is injective. Indeed, in that case, we have the following equivalences for $z\in\mathbb{C}^{n}$:

[TABLE]

[TABLE]

[TABLE]

[TABLE]

Let’s assume that $M$ is invertible. The relationship $UN+V\overline{M}=0$ is easy to prove. As for the injectivity, consider $z\in\mathbb{C}^{n}$ such that $z-N\overline{M}^{-1}\bar{z}=0$. Then $w:=M^{-1}z$ satisfies $Mw-N\overline{w}=0$, so that $M(iw)+N(\overline{iw})=0$. By assumption, the unique solution is $iw=0$, hence $z=0$. To treat the case where $M$ and $N$ are invertible, it suffices to multiply $U$ and $V$ by $N^{-1}$. ∎

3 Application to real linear system

Consider $m\in\mathbb{N}$, a matrix $A\in\mathcal{M}_{m}(\mathbb{R})$ that is positive definite, a vector $b\in\mathbb{R}^{m}$, and the linear system

[TABLE]

Thanks to a Gauss pivot, we may assume that $m=2n$ and view the system over the set of complex numbers. The first half of the variables can be viewed as real parts of complex variables, and the second half as their imaginary parts. To this end, define $B,C,D\in\mathcal{M}_{n}(\mathbb{R})$ and $z,p\in\mathbb{C}^{n}$ such that

[TABLE]

where $(\cdot)^{T}$ stands for transpose. System (38) is equivalent to

[TABLE]

where $M:=(B+D-i(C-C^{T}))/2\succ 0$ (positive definiteness will become clear in the following) and $N:=(B-D+i(C+C^{T}))/2$. Indeed, system (38) can be written as either of the three following equivalent systems:

$\frac{1}{2}\begin{pmatrix}B&C\\ C^{T}&D\end{pmatrix}\begin{pmatrix}z+\bar{z}\\ i(\bar{z}-z)\end{pmatrix}=b,$ 2. 2.

\left\{\begin{array}[]{rcl}\frac{B-iC}{2}z+\frac{B+iC}{2}\bar{z}&=&\Re p\\[5.0pt] \frac{C^{T}-iD}{2}z+\frac{C^{T}+iD}{2}\bar{z}&=&\Im p,\\ \end{array}\right. 3. 3.

$\frac{B+D-i(C-C^{T})}{2}z+\frac{B-D+i(C+C^{T})}{2}\bar{z}=p.$

Proposition 2 informs us that system (39) is in turn equivalent to

[TABLE]

As a result, the real linear system (38) is equivalent to the complex linear system (40). To the best of our knowledge, this relationship between real and complex linear systems has not been presented in past literature.

With regards to numerical aspects, an efficient method for solving the linear system (38) is the Cholesky factorization [2] which requires roughly $m^{3}/3=8n^{2}/3$ flops. It searches for a lower triangular matrix $L\in\mathbb{R}^{m\times m}$ such that $A=LL^{T}$. Next, it successively solves for $Ly=b,~{}y\in\mathbb{R}^{m}$ and $L^{T}x=y,~{}x\in\mathbb{R}^{m}$. Unfortunately, the matrix multiplication in the complex linear system (40) makes it more expensive to solve. However, the spectra of the positive definite matrices $\overline{M}\in\mathcal{M}_{n}(\mathbb{C})$ and $M-N\overline{M}^{-1}\overline{N}\in\mathcal{M}_{n}(\mathbb{C})$ are contractions of the spectrum of $A\in\mathcal{M}_{2n}(\mathbb{R})$. They hence have a better conditioning number (ratio of maximum to minimum eigenvalue) than $A$. This is relevant since a Cholesky factorization of both these matrices is required to solve the complex linear system (40). For work on the complex Cholesky factorization, see [7, 8].

We now clarify our statement regarding spectrum contraction. Given a Hermitian matrix $U\in\mathcal{M}_{n}(\mathbb{C})$, let $\lambda_{1}(U),\ldots,\lambda_{n}(U)$ denote its eigenvalues in increasing order. Given $\lambda\in\mathbb{R}$ and $z,w\in\mathbb{C}^{m}$, it holds that

[TABLE]

thereby equating the spectra of the two square matrices in the equation. Since $M$ is an extraction of the left-hand matrix, Cauchy’s interlace theorem [1, 5, 4, 3] implies that

[TABLE]

In addition, the interlacing property of the eigenvalues of the Schur complement of a Hermitian matrix [6, Theorem 5] implies that

[TABLE]

See [9] for a nice reference on the Schur complement. We now provide a numerical experiment. Consider the real linear system (38) with $m:=6$,

[TABLE]

and

[TABLE]

The data has been randomly generated with MATLAB R2015b. The eigenvalues of $A$ in increasing order are

[TABLE]

MATLAB R2015b yields the following solution to $Ax=b,~{}x\in\mathbb{R}^{m}$:

[TABLE]

Based on the discussion in this section, we may instead solve the complex linear system $(M-N\overline{M}^{-1}\overline{N})z=p-N\overline{M}^{-1}\overline{p},~{}z\in\mathbb{C}^{n},$ where

[TABLE]

and

[TABLE]

This yields

[TABLE]

to be compared with $x$ in (46).

The eigenvalues of $M-N\overline{M}^{-1}\overline{N}$ in increasing order are

[TABLE]

which can be seen to interlace those of $A$ in (45). As a byproduct, the conditioning number of $M-N\overline{M}^{-1}\overline{N}$, equal to $43.3843$, is less than that of $A$, equal to $177.3795$. We remind the reader that the closer to 1, the better the conditioning of a matrix.

4 Conclusion

We classify linear systems involving complex variables and their conjugates with respect to systems involving only complex variables. Our main tool is the orthogonal projection. As a result, we obtain a new link between linear systems with real variables and linear systems with complex variables. Our findings are illustrated on several numerical examples. A natural question follows: when can algebraic varieties defined by non-holomorphic polynomials be reduced to varieties defined by holomorphic polynomials? This manuscript gives the answer when the polynomials are of degree one.

Acknowledgements

I wish like to thank the anonymous reviewers for their helpful comments. Many thanks to Ximun Loyatho and Antoine Coutand for the fruitful discussions during my visit to Imswan.