On the global convergence of the Jacobi method for symmetric matrices of order 4 under parallel strategies

TL;DR

This paper proves the global convergence of certain parallel cyclic Jacobi methods for symmetric matrices of order 4, showing they consistently reduce the off-diagonal norm, and discusses the speed variability depending on matrix properties.

Contribution

It establishes the global convergence of specific parallel Jacobi strategies for 4x4 symmetric matrices, a result not previously confirmed for these methods.

Findings

The inequality S(A^{[2]}) ≤ (1 - 10^{-5}) S(A) holds for all symmetric 4x4 matrices after two cycles.

The method's convergence is guaranteed under all fully parallel strategies considered.

There exist matrices where the first cycle does not significantly reduce the off-diagonal norm, indicating variability in convergence speed.

Abstract

The paper analyzes special cyclic Jacobi methods for symmetric matrices of order $4$. Only those cyclic pivot strategies that enable full parallelization of the method are considered. These strategies, unlike the serial pivot strategies, can force the method to be very slow or very fast within one cycle, depending on the underlying matrix. Hence, for the global convergence proof one has to consider two or three adjacent cycles. It is proved that for any symmetric matrix $A$ of order~$4$ the inequality $S(A^{[2]})\leq(1-10^{-5})S(A)$ holds, where $A^{[2]}$ results from $A$ by applying two cycles of a particular parallel method. Here $S(A)$ stands for the Frobenius norm of the strictly upper-triangular part of $A$. The result holds for two special parallel strategies and implies the global convergence of the method under all possible fully parallel strategies. It is also proved that for every $\epsilon>0$ and $n\geq4$ there exist a symmetric matrix $A(\epsilon)$ of order $n$ and a cyclic strategy, such that upon completion of the first cycle of the appropriate Jacobi method the inequality $S(A^{[1]})> (1-\epsilon)S(A(\epsilon))$ holds.

Equations

$$A^{(k+1)}=R_k^T{A}^{(k)}{R}_k,k\ge 0;A^{(0)}=A,$$

$$\tan 2{\phi }^{(k)}=\frac{2a_{ij}^{(k)}}{a_{ii}^{(k)}-a_{jj}^{(k)}},{\phi }^{(k)}\in [-\pi /4,\pi /4],$$

$$a_{ii}^{(k+1)}$$

$$a_{jj}^{(k+1)}$$

$$S^2(A^{(k+1)})=S^2(A^{(k)})-(a_{ij}^{(k)}{)}^2.$$

$$S(X)=\frac{\sqrt{2}}{2}\Vert X-\text{diag}(X){\Vert }_F=\sqrt{\sum _{i=1}^{n-1}\sum _{j=i+1}^n{x}_{ij}^2},X=X^T=(x_{ij}).$$

$$\underset{k\to \infty }{\lim }S(A^{(k)})=0.$$

$$S^2(A^{(\tau N)})\le \gamma S^2(A),0\le \gamma <1,\tau \in \{1,2,3\},N=\frac{n(n-1)}{2},$$

$$(i_r,j_r),(i_{r+1},j_{r+1})\to (i_{r+1},j_{r+1}),(i_r,j_r),$$

$$m_{ij}=m_{ji}=k,\text{if}I(k)=(i,j),i<j,$$

$$M_{I_{1}}=\left[\begin{array}[]{cccc}*&4&0&2\\ 4&*&3&1\\ 0&3&*&5\\ 2&1&5&*\\ \end{array}\right]\quad\text{and}\quad M_{I_{2}}=\left[\begin{array}[]{cccc}*&4&2&0\\ 4&*&1&3\\ 2&1&*&5\\ 0&3&5&*\\ \end{array}\right],$$

$$\mathbf{M}_{1}=\left[\begin{array}[]{cccc}*&2&0&1\\ 2&*&1&0\\ 0&1&*&2\\ 1&0&2&*\\ \end{array}\right]\quad\text{and}\quad\mathbf{M}_{2}=\left[\begin{array}[]{cccc}*&2&1&0\\ 2&*&0&1\\ 1&0&*&2\\ 0&1&2&*\\ \end{array}\right],\ \text{respectively}.$$

$${\mathcal{O}}_1$$

$${\mathcal{O}}_1^{\prime }$$

$${\mathcal{O}}_1^{\prime \prime }$$

$$M_2=P^T{M}_{1}P,$$

$$A$$

$$\stackrel{(1,4),(2,3)}{⟶}$$

$$S(A^{[3]})=S(A^{(3N)})=S(A^{(18)})\le S(A^{(16)})\le \gamma S(A^{(4)})\le S(A),$$

$${\mathcal{O}}_1=(1,3),(2,4),(1,4),(2,3),(1,2),(3,4).$$

$$A=\left[\begin{array}[]{cccc}a_{11}&0&a_{13}&a_{14}\\ 0&a_{22}&a_{23}&a_{24}\\ a_{13}&a_{23}&a_{33}&0\\ a_{14}&a_{24}&0&a_{44}\\ \end{array}\right].$$

$$Q=[e_{1}\ e_{3}\ e_{4}\ {-}e_{2}]=\left[\begin{array}[]{rrrr}1&0&0&0\\ 0&0&0&-1\\ 0&1&0&0\\ 0&0&1&0\\ \end{array}\right].$$

$$Q^{T}XQ={\small\left[\begin{array}[]{rrrr}x_{11}&x_{13}&x_{14}&-x_{12}\\ x_{31}&x_{33}&x_{34}&-x_{32}\\ x_{41}&x_{43}&x_{44}&-x_{42}\\ -x_{21}&-x_{23}&-x_{24}&x_{22}\\ \end{array}\right]},\qquad QXQ^{T}={\small\left[\begin{array}[]{rrrr}x_{11}&-x_{14}&x_{12}&x_{13}\\ -x_{41}&x_{44}&-x_{42}&-x_{43}\\ x_{21}&-x_{24}&x_{22}&x_{23}\\ x_{31}&-x_{34}&x_{32}&x_{33}\\ \end{array}\right]}.$$

$${\mathcal{T}}_A(H)=(R(1,3,\varphi )R(2,4,\psi )Q{)}^{T}HR(1,3,\varphi )R(2,4,\psi )Q,H\in {\mathcal{S}}_4,$$

$${\mathcal{T}}^k(A)$$

$$S({\mathcal{T}}^{k+1}(A))\le S({\mathcal{T}}^k(A)),k\ge 0.$$

$$\mathcal{T}:\left[\begin{array}[]{cccc}a_{11}&0&a_{13}&a_{14}\\ 0&a_{22}&a_{23}&a_{24}\\ a_{13}&a_{23}&a_{33}&0\\ a_{14}&a_{24}&0&a_{44}\\ \end{array}\right]\mapsto\left[\begin{array}[]{cccc}a_{11}^{\prime}&0&a_{13}^{\prime}&a_{14}^{\prime}\\ 0&a_{22}^{\prime}&a_{23}^{\prime}&a_{24}^{\prime}\\ a_{13}^{\prime}&a_{23}^{\prime}&a_{33}^{\prime}&0\\ a_{14}^{\prime}&a_{24}^{\prime}&0&a_{44}^{\prime}\\ \end{array}\right],$$

$$a_{11}^{\prime }$$

$$a_{22}^{\prime }$$

$$a_{33}^{\prime }$$

$$a_{44}^{\prime }$$

$$\tan (2\varphi )=\frac{2a_{13}}{a_{11}-a_{33}},\tan (2\psi )=\frac{2a_{24}}{a_{22}-a_{44}}.$$

$${\mathcal{T}}^k(A)=(Q^k{)}^T{A}^{(2k)}{Q}^k,k\ge 0.$$

Full text

Erna Begović Kovač, Faculty of Chemical Engineering and Technology, University of Zagreb, Marulićev trg 19, 10000 Zagreb, Croatia

Vjeran Hari, Department of Mathematics, Faculty of Science, University of Zagreb, Bijenička 30, 10000 Zagreb, Croatia

This work has been fully supported by Croatian Science Foundation under the project 3670.

1. Introduction

The Jacobi method applies a sequence of similarity transformations by plane rotations to a symmetric matrix in order to diagonalize it. The method can be described as an iterative process of the for

[TABLE]

where $R_{k}$ are plane rotations and $A$ is a symmetric matrix of order $n$. The method is globally convergent if, for each starting $A$, the generated sequence $(A^{(k)})$ converges to a diagonal matrix. Its global (asymptotic) convergence has been considered in [8, 9, 3, 12, 17] ([19, 10]) and its accuracy in [4, 5, 6, 15]. A one-sided version of the method has been studied in [13, 18] and the block versions in [7, 11, 1]. There are many papers on Jacobi methods, and further references can be found within the bibliographies of the papers cited above.

At the step $k$ the method annihilates two off-diagonal elements of $A^{(k)}$, $a_{i(k)j(k)}^{(k)}$ and $a_{j(k)i(k)}^{(k)}$, $i(k)<j(k)$. The element $a_{i(k)j(k)}^{(k)}$ is the pivot element while $i=i(k)$ and $j=j(k)$ are pivot indices. The way of selecting the pivot pair at each step is called pivot strategy. The elements of $R_{k}$ are the same as in the identity matrix $I_{n}$, except for the elements at positions $(i,i)$, $(i,j)$, $(j,i)$, $(j,j)$, which are $\cos\varphi^{(k)}$, $-\sin\varphi^{(k)}$, $\sin\varphi^{(k)}$, $\cos\varphi^{(k)}$, respectively. The rotation angle is determined by the known formula

[TABLE]

which implies

[TABLE]

and

[TABLE]

Here $S(X)$ stands for the off-norm of a symmetric matrix $X$ of order $n$,

[TABLE]

In the definition (1.1) of the rotation angle, we assume that $\varphi^{(k)}=0$ if $a_{ij}^{(k)}=0$ and $a_{ii}^{(k)}=a_{jj}^{(k)}$. It is the most natural assumption which can be rephrased as: if the pivot element is zero, just skip it.

Since the diagonal elements converge if the rotation angle is chosen as in the relation (1.1) (see [14]), it is easy to show that the obtained sequence $(A^{(k)})$ converges to some diagonal matrix if and only if

[TABLE]

Therefore, the method is globally convergent if (1.4) holds for any initial $A$. Since the sequence $(S(A^{(k)}))$ is nonincreasing, for the global convergence of the method it is sufficient to show that for any symmetric matrix $A$ we have

[TABLE]

where $\gamma$ and $\tau$ do not depend on $A$. Here we prove that the relation (1.5) holds with $\tau=2$ or $\tau=3$, for the case $n=4$ and for those cyclic strategies which enable parallel processing. For these strategies one cycle (or sweep) consists of three “parallel steps”.

Why would one consider the Jacobi method for symmetric matrices of order $4$ when that problem can be solved directly? Jacobi method is known for its high relative accuracy on well behaved symmetric matrices, for its efficiency on nearly diagonal matrices and for its suitability for parallel processing. So, the natural choice of a pivot strategy for matrices of order $4$ is a parallel strategy. We have discovered that parallel strategies are very special. Depending on the underlying matrix, the reduction of the quantity $S(A)$ per sweep can be extremely slow or fast. This knowledge can be used to improve the implementation of the algorithm. Finally, the Jacobi method for large symmetric positive definite matrices is nowadays implemented as one sided block algorithm. At each step the block algorithm has to solve the same eigenvalue problem but for much smaller matrix, typically of order $16$–$256$. For this purpose one can use an element-wise Jacobi method or one can accelerate it by using the block algorithm which solves a $4$ by $4$ eigenvalue problem at each step.

There are several comments related to the inequality (1.5) and its proof. First, the proof presented here reveals that the reduction of the quantity $S(A)$ during one cycle can be arbitrary small. It sheds light to convergence failure of the cyclic Jacobi method discussed in [3]. We show that for every $\epsilon>0$ there is a starting matrix $A(\epsilon)$ and a cyclic Jacobi method such that upon completion of the first cycle the inequality $S(A^{[1]})>(1-\epsilon)S(A(\epsilon))$ holds. This fact is first proved for $n=4$ and then for any $n\geq 4$. Hence the global convergence consideration for the general cyclic Jacobi method should scrutinize more than one cycle of the process. Second, the presented result covers the most difficult part in the proof that every cyclic Jacobi method for symmetric matrices of order $4$ is globally convergent [1, 2].

The paper is divided into five sections and three appendices. In Section 2 we introduce notation and the basic concepts of the theory of equivalent strategies. We also recall some known convergence results. In Section 3 we concentrate on parallel strategies and introduce an auxiliary tool, a linear operator $\mathcal{T}_{A}$, which simplifies the convergence analysis. The convergence result is formulated and proved for some trivial cases. Section 4 is devoted to the global convergence proof and Section 5 to the construction of the above mentioned matrix $A(\epsilon)$ and to the proofs of the related results. Since the proofs of the main results are pretty complicated, we have moved all lengthy and technical proofs to appendices A, B and C. They are related to the results from sections 3, 4 and 5, respectively.

Some of the results presented here can be found in the unpublished thesis [1].

2. Basic concepts and notation

For the Jacobi method for symmetric matrices of order $n$, the pivot strategy can be defined as a function $I:\mathbb{N}_{0}\rightarrow\mathbf{P}_{n}$, where $\mathbb{N}_{0}=\{0,1,2,3,\ldots\}$ and \mathbf{P}_{n}=\big{\{}(i,j)\ \big{|}\ 1\leq i<j\leq n\big{\}}. We say that at step $k$, $I$ selects the pivot pair $I(k)=(i(k),j(k))$ which lies in $\mathbf{P}_{n}$. Let $I$ be a pivot strategy. If there is a positive integer $T$ such that $I(k+T)=I(k)$ for all $k\geq 0$, we say that $I$ is periodic with period $T$. If $T=N\equiv\frac{n(n-1)}{2}$ and \{I(k)\ \big{|}\ 0\leq k\leq T-1\}=\mathbf{P}_{n}, the pivot strategy is cyclic.

For $S\subseteq\mathbf{P}_{n}$, let \mathcal{\mbox{\Large\mathcal{O}}}(S) denote the set of all finite sequences made of the elements of $S$, assuming that each pair from $S$ appears at least once in each sequence from \mathcal{\mbox{\Large\mathcal{O}}}(S). Let $\mathcal{O}$ be a sequence of pairs from \mathcal{\mbox{\Large\mathcal{O}}}(S). An admissible transposition on $\mathcal{O}$ is any transposition of two adjacent pairs from $\mathcal{O}$,

[TABLE]

provided that $\{i_{r},j_{r}\}\cap\{i_{r+1},j_{r+1}\}=\emptyset$. For such pairs we say that they commute, or that they are disjoint. Two sequences \mathcal{O},\mathcal{O}^{\prime}\in\mathcal{\mbox{\Large\mathcal{O}}}(S) are called

• (i)

Equivalent if one can be obtained from the other by a finite number of admissible transpositions. Then we write $\mathcal{O}\sim\mathcal{O}^{\prime}$.

• (ii)

Shift-equivalent if $\mathcal{O}=[\mathcal{O}_{1},\mathcal{O}_{2}]$ and $\mathcal{O}^{\prime}=[\mathcal{O}_{2},\mathcal{O}_{1}]$, where $[\mathcal{O}_{1},\mathcal{O}_{2}]$ stands for the concatenation of the sequences $\mathcal{O}_{1}$ and $\mathcal{O}_{2}$. We write $\mathcal{O}\stackrel{{\scriptstyle s}}{{\sim}}\mathcal{O}^{\prime}$.

• (iii)

Weakly equivalent if one can find $\mathcal{O}_{1},\ldots,\mathcal{O}_{r-1}$ from \mathcal{\mbox{\Large\mathcal{O}}}(S) such that in the sequence $\mathcal{O}\equiv\mathcal{O}_{0},\mathcal{O}_{1},\ldots,\mathcal{O}_{r}\equiv\mathcal{O}^{\prime}$, each pair of adjacent terms $\mathcal{O}_{i}$, $\mathcal{O}_{i+1}$, $0\leq i\leq r-1$, consists of either equivalent or shift-equivalent terms. In such a case, we write $\mathcal{O}\stackrel{{\scriptstyle w}}{{\sim}}\mathcal{O}^{\prime}$.

One can check that $\sim$, $\stackrel{{\scriptstyle s}}{{\sim}}$ and $\stackrel{{\scriptstyle w}}{{\sim}}$ are equivalence relations on \mathcal{\mbox{\Large\mathcal{O}}}(S). In our application we shall have $S=\mathbf{P}_{n}$.

Once these equivalence relations are defined on \mathcal{\mbox{\Large\mathcal{O}}}(\mathbf{P}_{n}), they can easily be transferred to the set of cyclic pivot strategies. Here is the procedure.

Let $I$ be a cyclic pivot strategy. By $\mathcal{O}_{I}$ we mean the sequence of pairs $I(0)$, $I(1)$, $\>\ldots\>$, $I(N-1)$. Conversely, for \mathcal{O}\in\mathcal{\mbox{\Large\mathcal{O}}}(\mathbf{P}_{n}), $\mathcal{O}=(i_{0},j_{0}),(i_{1},j_{1}),\ldots,(i_{N-1},j_{N-1})$, the cyclic strategy generated by $\mathcal{O}$ is defined by $I_{\mathcal{O}}(k)=(i_{\omega(k)},j_{\omega(k)})$, provided that $k\equiv\omega(k)\ (\mathrm{mod}\ N)$, $0\leq\omega(k)\leq N-1$, $k\geq 0$. In other words, $I_{\mathcal{O}}(k)$ runs through $\mathcal{O}$ in the cyclic way as $k$ increases.

Two cyclic strategies $I$ and $I^{\prime}$ are equivalent (we write $I\sim I^{\prime}$), shift-equivalent ($I\stackrel{{\scriptstyle s}}{{\sim}}I^{\prime}$) and weakly equivalent ($I\stackrel{{\scriptstyle w}}{{\sim}}I^{\prime}$) if the same is true for the corresponding sequences $\mathcal{O}_{I}$ and $\mathcal{O}_{I^{\prime}}$. Note that for the shift-equivalent strategies we have $I^{\prime}(k)=I(k+\sigma)$, $k\geq 0$, for some shift $\sigma$, $0\leq\sigma\leq N-1$. (We can confine to nonnegative shifts since $I(k-\sigma)=I(k+N-\sigma)$.)

The importance of weakly equivalent cyclic strategies comes from the following result.

Theorem 2.1**.**

[17]* If the Jacobi method converges for some cyclic strategy $I$, then it also converges for all strategies that are weakly equivalent to $I$.*

Note that Theorem 2.1 also covers the cases of equivalent and shift-equivalent strategies. Another important result regarding the convergence under two weakly equivalent strategies is proved in [11, Lemma 4.8].

A cyclic strategy $I$ can be represented by the matrix $M_{I}=(m_{ij})$, where

[TABLE]

and $m_{ss}=-1$, $1\leq s\leq n$. Instead of $-1$, we shall display $*$ to indicate that the diagonal positions are not part of the pivot sequence (see (3.1)). If $I=I_{\mathcal{O}}$, we shall also write $M_{\mathcal{O}}$.

3. Parallel strategies in the case $n=4$

Let $A$ be a symmetric matrix of order $4$. Since the length of each \mathcal{O}\in\mathcal{\mbox{\Large\mathcal{O}}}(\mathbf{P}_{4}) equals $\frac{4\cdot 3}{2}=6$, each cyclic Jacobi method applies six steps within one cycle. Among all cyclic strategies a distinguished role is played by the “parallel” ones. They enable parallel processing, so the corresponding method will be called parallel Jacobi method (cf. [16]). Each parallel Jacobi method for symmetric matrices of order $4$ applies three parallel steps within each cycle. Every parallel step consists of two consecutive steps which can be performed concurrently. This way, instead of six sequential steps, using a parallel pivot strategy, we apply three parallel steps within one cycle.

As we shall see, it will be sufficient to study just two cyclic pivot strategies $I_{1}$ and $I_{2}$, which have the following two-dimensional representations

[TABLE]

respectively. All cyclic strategies that can be fully parallelized, are shift equivalent to $I_{1}$ or $I_{2}$. Therefore, the convergence results for all parallel strategies follow from the results for the strategies $I_{1}$ and $I_{2}$.

Consider the sets of pairs $\{(1,3),(2,4)\}$, $\{(1,4),(2,3)\}$, $\{(1,2),(3,4)\}$. Note that the pairs within braces commute. These are the only sets that contain commuting pairs and only they can define parallel Jacobi steps. From the first braces we see that the corresponding plane rotations $R(1,3,\varphi_{13})$ and $R(2,4,\varphi_{24})$ commute, and also their entries can be computed independently of each other. So, the corresponding Jacobi steps can be applied in parallel: first apply concurrently the left transformations and then the right ones, or vice versa. This corresponds to the one parallel step which consists of two subsequent ordinary Jacobi steps. The same can be said for the steps corresponding to the other two braces. This leads us to parallel strategies, which we represent by the matrices

[TABLE]

Here, the matrix entries count the parallel steps and mark the pivot positions associated with them.

By inspecting all commuting pairs, we conclude that there are exactly six parallel strategies and they can be grouped into two clusters which are actually equivalent classes for the relation $\stackrel{{\scriptstyle s}}{{\sim}}$. They are defined by the following orderings from \mathcal{\mbox{\Large\mathcal{O}}}(\mathbf{P}_{4}), where $\mathcal{O}_{j}\stackrel{{\scriptstyle s}}{{\sim}}\mathcal{O}_{j}^{\prime}\stackrel{{\scriptstyle s}}{{\sim}}\mathcal{O}_{j}^{\prime\prime}$, $j=1,2$,

[TABLE]

In order to prove the global convergence of the Jacobi method under all six parallel strategies, it is sufficient to prove it for the strategies $I_{1}$ and $I_{2}$. This follows from Theorem 2.1. Next, we show that the strategies $I_{1}$ and $I_{2}$ are closely connected, so that the method converges under one of them if and only if it converges under the other one. To this end, note that the matrices $\mathbf{M}_{1}$ and $\mathbf{M}_{2}$ are permutationally similar,

[TABLE]

where $P=P_{12}$ or $P=P_{34}$. Here $P_{ij}$ is the transposition which interchanges rows (columns) $i$ and $j$ if a matrix is premultiplied (postmultiplied) by it. If (3.2) holds, we say that $I_{2}$ and $I_{1}$ are permutationally equivalent (see [1, 2]).

Proposition 3.1**.**

Let $A=(a_{ij})$ be a symmetric matrix of order $4$. Let $A^{(0)}=A,A^{(1)},\ldots$ be obtained by applying the cyclic Jacobi method defined by the strategy $I_{2}$ on $A$. Let $P=P_{12}$ or $P=P_{34}$, and let $\mathsf{A}^{(0)}=P^{T}AP,\mathsf{A}^{(1)},\ldots$ be obtained by applying the cyclic Jacobi method defined by the strategy $I_{1}$ on $\mathsf{A}^{(0)}$. Then $\mathsf{A}^{(2r)}=P^{T}A^{(2r)}P$, $r\geq 0$.

Proof.

The proof has been moved to A. ∎

Thus, Proposition 3.1 implies that the Jacobi method converges under the strategy $I_{2}$ if and only if it converges under the strategy $I_{1}$. In particular, if the relation (1.5) holds for the method defined by $I_{1}$, with some $\tau$ and $\gamma$, it holds for the method defined by $I_{2}$ with the same $\tau$ and $\gamma$, and vice versa. Theorem 3.4 below, shows that the relation (1.5) holds for the strategy $I_{1}$ with $\tau=2$ and $\gamma=1-10^{-5}$.

What can be said for the method under the strategies $I_{\mathcal{O}_{1}^{\prime}}$, $I_{\mathcal{O}_{1}^{\prime\prime}}$ and $I_{\mathcal{O}_{2}^{\prime}}$, $I_{\mathcal{O}_{2}^{\prime\prime}}$? For these strategies, the relation (1.5) holds with the same $\gamma$ and with $\tau$ larger for $1$. In particular, for $\tau=3$ and $\gamma=1-10^{-5}$. We shall show it for $I_{\mathcal{O}_{1}^{\prime\prime}}$. For the other three strategies the proof is similar.

Let us apply the Jacobi method defined by the strategy $I_{\mathcal{O}_{1}^{\prime\prime}}$ to a symmetric matrix $A$ of order $4$, thus generating the sequence of matrices $A^{(0)}=A$, $A^{(1)}$, $A^{(2)},\ldots{}$. Let us consider $\tau+1=3$ cycles of the method. We display each second iterate, i.e. the iterates obtained after each of the first nine parallel steps:

[TABLE]

We concentrate on the matrix $A^{(4)}$. If another Jacobi method is applied to $A^{(4)}$, the one defined by the strategy $I_{1}$, one obtains (after each two steps) the same matrices $A^{(6)},A^{(8)},A^{(10)},\ldots{}$. After two sweeps, one obtains the matrix $A^{(16)}$ and, if the relation (1.5) holds for $I_{1}$ with $\tau=2$ and $\gamma<1$, then one has $S(A^{(16)})\leq\gamma S(A^{(4)})$. Therefore, one obtains

[TABLE]

proving the claim.

3.1. The cyclic strategy $I_{1}$

We focus on strategy $I_{1}=I_{\mathcal{O}_{1}}$ where

[TABLE]

By this strategy, at the beginning of each cycle (except for the first cycle), the elements at the positions $(1,2)$ and $(3,4)$ are zero. Since we consider the global convergence, we can assume that the initial matrix already has the form

[TABLE]

Let $[e_{1}\ e_{2}\ e_{3}\ e_{4}]$ denote the column partition of the identity matrix, and let

[TABLE]

The similarity transformation with $Q$ and with $Q^{T}$ has the following effect on the elements of a square matrix $X=(x_{rs})$,

[TABLE]

Thus, for each $A$ we have $S(Q^{T}AQ)=S(A)$. We see a favorable movement of the elements lying at the pivot positions for the parallel steps. We can use it to define a new iterative process, closely related to the original Jacobi process, where the pivot elements always remain at the same positions. This will simplify the analysis.

Therefore, we introduce a linear operator which is comprised of the transformation corresponding to the first parallel step under $I_{1}$ followed by the similarity transformation with $Q$.

Definition 3.2**.**

Let $\mathcal{S}_{4}$ denote the vector space of real $4$ by $4$ symmetric matrices. For $A\in\mathcal{S}_{4}$ let

[TABLE]

where $R(1,3,\phi)$ and $R(2,4,\psi)$ are Jacobi rotations which annihilate the elements $a_{13}$ and $a_{24}$ of $A$, respectively, and $Q$ is defined by the relation (3.4). The rotation angles $\phi$, $\psi$ are from the interval $[-\frac{\pi}{4},\frac{\pi}{4}]$, so that the formulas (1.1)–(1.3) hold.

For $A\in\mathcal{S}_{4}$ let $\mathcal{T}(A)=\mathcal{T}_{A}(A)$ and for any $k\geq 0$

[TABLE]

Thus, $\mathcal{T}_{A}:\mathcal{S}_{4}\mapsto\mathcal{S}_{4}$ is a linear operator. Note that if $a_{13}=0$ and $a_{24}=0$, then $\mathcal{T}_{A}$ reduces to the similarity transformation with the similarity matrix $Q$. The function $\mathcal{T}$ is not linear. However, it satisfies

[TABLE]

If $A\in\mathcal{S}_{4}$ is as in the relation (3.3) and $A^{\prime}=\mathcal{T}(A)$, then we have

[TABLE]

with

[TABLE]

The rotation angles $\phi\in[-\frac{\pi}{4},\frac{\pi}{4}]$ and $\psi\in[-\frac{\pi}{4},\frac{\pi}{4}]$ are determined by

[TABLE]

First, we show that the repeated application of $\mathcal{T}$ to $A$ yields the matrices which are closely related to Jacobi iterations under the parallel strategy $I_{1}$.

Proposition 3.3**.**

Let $A\in\mathcal{S}_{4}$ and let $A^{(2k)}$ be obtained by applying $2k$ steps of the Jacobi method under the strategy $I_{1}$ to $A$. Then

[TABLE]

Proof.

The proof is lengthy and technical, so we have moved it to A. ∎

In particular, the relation (3.8) implies

[TABLE]

We use Proposition 3.3 to simplify the proof of the main result which follows.

Theorem 3.4**.**

Let $A\in\mathcal{S}_{4}$ be such that $a_{12}=0$, $a_{34}=0$ and let $A^{(12)}$ be obtained by applying $12$ steps of the Jacobi method under the strategy $I_{1}$ to $A$. Then

[TABLE]

with $\epsilon=10^{-5}$.

Note that $12$ steps correspond to two sweeps of the method. Theorem 3.4 ensures the global convergence of the method since the sequence of iterates $(S(A^{(l)}),l\geq 0)$ is nonincreasing and its subsequence $(S(A^{(12t)}),t\geq 0)$ converges to zero.

The proof of the main theorem is lengthy, hence we will devote the entire Section 4 to it. However, we first provide a lemma that covers the special cases when more than two off-diagonal elements are equal to zero. Then the relation (3.10) holds with much larger $\epsilon$ ($\epsilon=1$ or $1/2$).

Lemma 3.5**.**

Let $A=(a_{rs})\in\mathcal{S}_{4}$ be such that $a_{12}=0$ and $a_{34}=0$. If

• (i)

$a_{14}=0$* and $a_{23}=0$, then $A^{(2)}$ is diagonal.*

• (ii)

$a_{13}=0$* and $a_{24}=0$, then $A^{(4)}$ is diagonal.*

• (iii)

$a_{13}=0$, then $S^{2}(A^{(4)})\leq\frac{1}{2}S^{2}(A)$.

• (iv)

$a_{24}=0$, then $S^{2}(A^{(4)})\leq\frac{1}{2}S^{2}(A)$.

• (v)

$a_{14}=0$, then $S^{2}(A^{(4)})\leq\frac{1}{2}S^{2}(A)$.

• (vi)

$a_{23}=0$, then $S^{2}(A^{(4)})\leq\frac{1}{2}S^{2}(A)$.

Proof.

The proof has been moved to A. ∎

4. Proof of Theorem 3.4

Let $\epsilon=10^{-5}$. Then the assertion (3.10) of Theorem 3.4 can be expressed in the form

[TABLE]

Instead of working with matrices $A^{(l)}$, $0\leq l\leq 12$, we shall work with $B^{(k)}=\mathcal{T}^{k}(A)=(b_{rs}^{(k)})$, $0\leq k\leq 6$. Let $B\equiv B^{(0)}$, so that $B=A$ holds. If $S(B)=0$, then Theorem 3.4 holds. We assume $S(B)>0$.

Contrary to the assertion of the theorem suppose that

[TABLE]

We shall show that the relation (4.2) leads to a contradiction.

From Lemma 3.5, we conclude that all off-diagonal elements of $B^{(k)}$ except for $b_{12}^{(k)}$ and $b_{34}^{(k)}$ are non-zero for $0\leq k\leq 4$. Furthermore, by Lemma 3.5(i), we have $|b_{14}^{(5)}|+|b_{23}^{(5)}|>0$, and since $S(B^{(k)})\leq S(B^{(k-1)})$, $k\geq 1$, we have

[TABLE]

Let

[TABLE]

From our assumptions it follows that $\delta_{k}>0$ for at least $0\leq k\leq 4$. Note that

[TABLE]

This implies

[TABLE]

in particular

[TABLE]

Formulas (3.6) describe the transition from $B^{(k-1)}$ to $B^{(k)}$ for any $k\geq 1$. In this transition we denote the angles $\phi$ and $\psi$ by $\phi_{k}$ and $\psi_{k}$, respectively. If we set $c_{\phi_{k}}=\cos\phi_{k}$, $c_{\psi_{k}}=\cos\psi_{k}$, $s_{\phi_{k}}=\sin\phi_{k}$, $s_{\psi_{k}}=\sin\psi_{k}$, $k\geq 1$, then from the formulas (3.6) we have

[TABLE]

Here, $c_{\phi_{k}}^{2}c_{\psi_{k}}^{2}+s_{\phi_{k}}^{2}s_{\psi_{k}}^{2}$ has been bounded by $\frac{1}{2}$, as in the proof of Lemma 3.5(v). This implies

[TABLE]

Together with (4.6), the relation (4.7) yields

[TABLE]

Lemma 4.1**.**

Exactly one of the following two assertions holds:

• (a)

$|b_{14}^{(k)}+b_{23}^{(k)}|\leq\sqrt{2}\delta_{k+1}S(B)<2\sqrt{\epsilon}S(B)$,  $0\leq k\leq 4$,

• (b)

$|b_{14}^{(k)}-b_{23}^{(k)}|\leq\sqrt{2}\delta_{k+1}S(B)<2\sqrt{\epsilon}S(B)$,  $0\leq k\leq 4$.

Proof.

Suppose that the two inequalities in (a) hold for some $k$, $0\leq k\leq 4$. Then, because of the relations (4.5) and (4.3), we have

[TABLE]

and therefore

[TABLE]

Thus, the corresponding inequality in (b) cannot hold for that $k$. Similarly, if the two inequalities in (b) hold for some $0\leq k\leq 4$, then the corresponding inequality in (a) cannot be true.

Now, let us show that if $(a)$ holds for $k=0$, then it holds for all $0\leq k\leq 4$. From the relations (3.6) it follows

[TABLE]

The relation (4.11) implies

[TABLE]

for $k\geq 1$. Therefore, if the two inequalities in (a) hold for $k=0$, then the relation (4.9) will hold for $k=1,2,3,4,5$. For any of these $k$ the relation (4.10) also holds, proving that the inequalities in (b) cannot hold. Hence we can conclude that the both inequalities in (a) hold for $0\leq k\leq 4$.

Similarly, if the two inequalities in (b) hold for $k=0$, they hold for all $0\leq k\leq 4$ and then the inequalities in (a) do not hold. ∎

We continue to prove (4.1) under the assumption $(a)$. The case (b) will be addressed later.

4.1. The case $|b_{14}+b_{23}|\leq\sqrt{2}\delta_{1}S(B)$

Let us see what can be concluded for the rotation angles. Using $\phi$, $\psi$ for $\phi_{1}$, $\psi_{1}$, respectively, from the relations (3.6) one easily obtains

[TABLE]

Hence,

[TABLE]

We used the definition of $\delta_{1}$ from (4.4). In the same way, one obtains

[TABLE]

for $k\geq 1$. Using the relations (4.5), (4.3), the assumption (a) and Lemma 4.1, we conclude that the relation (4.14) implies

[TABLE]

for $1\leq k\leq 5$. Thus

[TABLE]

Here we used $\epsilon=10^{-5}$. In (4.15) we have the strict inequalities when $\delta_{k}>0$ and that is certainly true for $1\leq k\leq 4$.

Lemma 4.2**.**

For the angles $\phi_{k}$, $\psi_{k}$, $1\leq k\leq 5$, we have the following relations.

• (i)

One of the following two relations holds

[TABLE]

[TABLE]

• (ii)

$|\sin(\phi_{k}+\psi_{k})|\leq|\phi_{k}+\psi_{k}|\leq 2.222\delta_{k}$.

• (iii)

$|\tan\phi_{k}+\tan\psi_{k}|\leq 4.444\delta_{k}$.

• (iv)

If $t\in\{|\tan\phi_{k}|,\tan\psi_{k}|\}$, then

[TABLE]

• (v)

$\max\left\{|\cot 2\phi_{k}|,|\cot 2\psi_{k}|,|\cot 2\phi_{k}+\cot 2\psi_{k}|\right\}\leq 4.49\delta_{k}$.

• (vi)

$|\cot 2\phi_{k}+\tan\phi_{k}+\cot 2\psi_{k}+\tan\psi_{k}|\leq 10.08\delta_{k}^{2}\leq 0.0454\delta_{k}$.

• (vii)

$2\leq|\cot 2\phi_{k}+\tan\phi_{k}-\cot 2\psi_{k}-\tan\psi_{k}|\leq 2+20.15214\delta_{k}^{2}\leq 2+0.0907\delta_{k}$.

Proof.

The proof is technical and has been moved to B. ∎

From the proof, one can easily check that in the assertions of Lemma 4.2, the inequality signs $\leq$ and $\geq$ standing left to $\delta_{k}$ can be replaced by $<$ and $>$ respectively, provided that $\delta_{k}>0$ (which is true for $0\leq k\leq 4$).

Let

[TABLE]

Lemma 4.1(a) implies $\nu_{k}\leq\sqrt{2}\delta_{k+1}$, $0\leq k\leq 4$. From the relation (4.11) it follows

[TABLE]

Hence by Lemma 4.2(ii) we have

[TABLE]

The next step is bounding $\delta_{k}$, $k=0,1,2$, by simple functions of the subsequent $\delta_{k}$s.

Lemma 4.3**.**

The quantities $\delta_{k}$ satisfy the following inequalities

[TABLE]

Hence we obtain

[TABLE]

Proof.

The proof has been moved to B. ∎

Lemma 4.4**.**

For the pivot elements of $B^{(k)}$ we have:

• (i)

$|b_{13}^{(k)}-b_{24}^{(k)}|\leq\nu_{k-1}S(B)\leq 2.222^{k-1}\delta_{k-1}\delta_{k-2}\cdots\delta_{2}\delta_{1}\sqrt{2}\delta_{1}S(B),\ 2\leq k\leq 6$*. *

In particular,

[TABLE]

• (ii)

\big{(}b_{13}^{(k)}+b_{24}^{(k)}\big{)}^{2}=2\delta_{k}^{2}S^{2}(B)-\big{(}b_{13}^{(k)}-b_{24}^{(k)}\big{)}^{2},\ k\geq 0.* *

Hence,

[TABLE]

and in particular

[TABLE]

• (iii)

$|\cot 2\phi_{k}-\cot 2\psi_{k}|\geq 1.999894\delta_{k},\quad 3\leq k\leq 4$.

Proof.

The proof is technical and has been moved to B. ∎

We come to the main part of the proof. So far, we derived the restrictions on the angles and other quantities, which are expressed in the previous lemmas. The question arises whether the diagonal elements, which enter into the definition of the angles, can allow all those limitations.

Consider the quantities

[TABLE]

Each $b^{(k)}$ will be expressed in two ways. On the one hand, we use (3.6) and (B.3) to obtain

[TABLE]

Hence, by the assertions (vi) and (vii) of Lemma 4.2, for $1\leq k\leq 5$ we have

[TABLE]

On the other hand, we use (B.3) or (3.7) to obtain

[TABLE]

Therefore, we have

[TABLE]

Using Lemma 4.4(iii) we obtain

[TABLE]

Furthermore, using Lemma 4.2(v), (4.21) and Lemma 4.4 we can bound from the above the right hand side of the inequality (4.23) divided by $S(B)$. We obtain

[TABLE]

These inequalities hold for $2\leq k\leq 4$. Hence, for $k=2,3$ we have

[TABLE]

By inspecting the two cases, one checks that the case $k=3$ yields the contradiction with the relation (4.2) from the beginning of this proof. This is exactly what we need to prove the assertion (3.10) of the theorem. For $k=3$ the relation (4.24) becomes

[TABLE]

Using Lemma 4.4(ii) (actually the bound from (4.20)) the above relation implies

[TABLE]

From Lemma 4.3 we have $\delta_{1}<0.0388\delta_{3}$ and $\delta_{2}<0.0771\delta_{4}$. Moreover,

[TABLE]

Dividing the inequality (4.25) by $\delta_{3}\delta_{4}$ and using (4.26) and (4.18) we get the contradiction

[TABLE]

We used the bound $0.0045$ for $\delta_{3}$ and $\delta_{4}$, and $2\epsilon$ for $\delta_{3}^{2}$.

4.2. The case $|b_{14}-b_{23}|\leq\sqrt{2}\delta_{1}S(B)$

The proof is similar as in the case $|b_{14}+b_{23}|\leq\sqrt{2}\delta_{1}S(B)$. We follow the lines of the proof above and modify it where necessary.

The expression on the right-hand side in the relation (4.13) can easily be brought to different form. We obtain

[TABLE]

Then the relation (4.14) becomes

[TABLE]

This implies

[TABLE]

Lemma 4.2 has to be modified and we formulate it as a new lemma.

Lemma 4.5**.**

For the angles $\phi_{k}$, $\psi_{k}$, $1\leq k\leq 5$, we have the following relations.

• (i)

One of the following two relations holds

[TABLE]

[TABLE]

• (ii)

$|\sin(\phi_{k}-\psi_{k})|\leq|\phi_{k}-\psi_{k}|\leq 2.222\delta_{k}$.

• (iii)

$|\tan\phi_{k}-\tan\psi_{k}|\leq 4.444\delta_{k}$.

• (iv)

If $t\in\{|\tan\phi_{k}|,\tan\psi_{k}|\}$, then

[TABLE]

• (v)

$\max\left\{|\cot 2\phi_{k}|,|\cot 2\psi_{k}|,|\cot 2\phi_{k}-\cot 2\psi_{k}|\right\}\leq 4.49\delta_{k}$.

• (vi)

$|\cot 2\phi_{k}+\tan\phi_{k}-\cot 2\psi_{k}-\tan\psi_{k}|\leq 10.08\delta_{k}^{2}\leq 0.0454\delta_{k}$.

• (vii)

$2\leq|\cot 2\phi_{k}+\tan\phi_{k}+\cot 2\psi_{k}+\tan\psi_{k}|\leq 2+20.15214\delta_{k}^{2}\leq 2+0.0907\delta_{k}$.

Proof.

The proofs of these assertions are very similar or identical to the proofs of the corresponding assertions of Lemma 4.2. ∎

Instead of $\nu_{k}$, we work with $\nu_{k}^{-}$. The relation (4.12) and the assertion (ii) of Lemma 4.5 imply

[TABLE]

The statement of Lemma 4.3 does not have to be modified, but the proof needs minor changes. We have explained those changes in B under the title “Proof of Lemma 4.3 in the case $|b_{14}-b_{23}|\leq\sqrt{2}\delta_{1}S(B)$”.

Lemma 4.4 has to be modified.

Lemma 4.6**.**

For the pivot elements of $B^{(k)}$ we have:

• (i)

$|b_{13}^{(k)}+b_{24}^{(k)}|\leq\nu_{k-1}^{-}S(B)\leq 2.222^{k-1}\delta_{k-1}\delta_{k-2}\ldots\delta_{2}\delta_{1}\cdot\sqrt{2}\delta_{1}S(B),\ 2\leq k\leq 6$*. *

In particular,

[TABLE]

• (ii)

\big{(}b_{13}^{(k)}-b_{24}^{(k)}\big{)}^{2}=2\delta_{k}^{2}S^{2}(B)-\big{(}b_{13}^{(k)}+b_{24}^{(k)}\big{)}^{2},\ \text{for}\ k\geq 0*. *

Hence,

[TABLE]

and in particular

[TABLE]

• (iii)

$|\cot 2\phi_{k}+\cot 2\psi_{k}|\geq 1.999894\delta_{k},\quad 3\leq k\leq 4$.

Proof.

The proof is similar to the proof of Lemma 4.4. We have moved it to B. ∎

To prove the main assertion (4.1) we use the same $b^{(k)}$ as earlier. The assertions (vi) and (vii) of Lemma 4.5 yield

[TABLE]

Using (4.22) we obtain

[TABLE]

Using Lemma 4.6(iii), the left-hand side can be bounded from below by $1.999894\cdot\delta_{k+1}\left|b_{13}^{(k)}-b_{24}^{(k)}\right|$ and for the case $k=3$ one can use Lemma 4.6(ii) to further reduce it to $2.828\delta_{3}\delta_{4}S(B)$.

Using (4.29), (4.28), Lemma 4.5(v), and Lemma 4.6(i), the right-hand side divided by $S(B)$ can be bounded from above by

[TABLE]

For $k=3$, after dividing by $S(B)$, one obtains

[TABLE]

which is the same inequality as (4.25). The rest of the proof is the same as earlier. ∎

At this point we would like to make a few comments.

• •

Theorem 3.4 obviously holds with somewhat larger $\epsilon$, e.g. one can try to complete the proof with $\epsilon=10^{-4}$. On the other hand, a small $\epsilon$ from the proof exposes the possibility of the very small reduction of $S(A)$ within one cycle. This happens when an underlaying matrix has a special structure. The next section deals with this issue.

• •

Although $1-10^{-5}$ is a small decrease of the off-norm within two cycles, the result does not mean that the convergence of the method should be slow. The proof is concentrated on the worst case scenario. Typically, the slower the method is within one cycle, the faster it is in the next cycle. Example 5.1 indicates that behavior.

• •

In this convergence proof we have explicitly used the diagonal elements of $A$, which is unusual when the reduction of $S(A)$ is considered. Usually, only the off-diagonal elements and the bounds on rotation angles are used (e.g. [12, 17, 19, 10]). In that case the proof is valid for a more general iterative process used in the global convergence analysis of Jacobi-type processes which use nonorthogonal transformation matrices [11, 1].

5. The slow off-norm reduction within one cycle

As it can be seen from the above theory, the decrease of the off-norm after one cycle of the Jacobi method under the strategy $I_{1}$ can be small. Here we give an example from [1], where the relative decrease of the off-norm after one cycle is less then $10^{-50}$.

Example 5.1**.**

Let

[TABLE]

with $\epsilon=10^{-52}$, $p_{1}=\epsilon$, $p_{2}=\epsilon\sqrt{\epsilon}$.

We have used MATLAB Symbolic Math Toolbox, in particular the Variable-precision arithmetic with $2l$ digits, to compute the matrix iterates under the cyclic Jacobi method defined by the strategy $I_{1}$. We display the off-norm of each iterate to $l$ significant digits. For $l=50$ we obtain

[TABLE]

As we can see from the table below, during the first cycle the off-norm of $H$ does not change in the first $50$ decimal places. But later it drops rapidly, especially in the $8$th step.

[TABLE]

In general, one can always find a matrix $A(\epsilon)$ such that the decrease of the off-norm after one cycle of the Jacobi method under the strategy $I_{1}$ is arbitrary small and depends only on $\epsilon$.

Proposition 5.2**.**

Let $0<\epsilon\leq 10^{-5}$,

[TABLE]

and let the cyclic Jacobi method defined by the strategy $I_{1}$ be applied to $H(\epsilon)$, thus generating the matrices $H^{(0)}=H(\epsilon),H^{(1)},\ldots{}$. After completing one full sweep we have

[TABLE]

Proof.

The proof is lengthy and technical, so it has been moved to C. ∎

We end the paper with the following important theorem.

Theorem 5.3**.**

For every $0<\epsilon<1$ and $n\geq 4$, there exists a symmetric matrix $A(\epsilon)$ of order $n$, depending on $\epsilon$ and a cyclic strategy $I$, such that

[TABLE]

Here $A^{(0)}=A(\epsilon)$ and $A^{(N)}$ is obtained from $A^{(0)}$ by applying a full cycle of the Jacobi method under the strategy $I$.

Proof.

Let $n=4$, $0<\epsilon<10^{-5}$, $\epsilon^{\prime}=(2\epsilon-\epsilon^{2})/17$ and $A(\epsilon)=H(\epsilon^{\prime})$, where $H(\epsilon^{\prime})$ is from Proposition 5.2. Let $I=I_{1}=I_{\mathcal{O}_{1}}$. Proposition 5.2 yields to $A^{(0)}=A(\epsilon)$,

[TABLE]

Since

[TABLE]

the proof is completed in this case.

If $10^{-5}\leq\epsilon<1$, then $1-10^{-5}\geq 1-\epsilon>0$. Hence, we can choose $A(\epsilon)=H(\frac{2\cdot 10^{-5}-10^{-10}}{17})$ to obtain $S(A^{(6)})>(1-10^{-5})S(A)\geq(1-\epsilon)S(A)$.

Let $n>4$ and let

[TABLE]

be a symmetric matrix of order $n$ with the following properties.

• (i)

$A_{11}^{[0]}$ is of order $4$ such that $S(A_{11}^{(6)})>(1-\epsilon)S(A_{11}^{[0]})$ holds when one full cycle of the Jacobi method under the strategy $I_{1}$ is applied to $A_{11}^{[0]}$. This follows from (5.1) because we have proved the theorem for $n=4$.

• (ii)

The block $A_{22}^{[0]}$ is diagonal.

The pivot strategy $I$ is defined by $I=I_{\mathcal{O}}$, $\mathcal{O}=[\mathcal{O}_{1},\mathcal{O}_{22},\mathcal{O}_{12}]$, where $\mathcal{O}_{22}$ is any ordering of the set $\mathcal{S}_{22}=\{(5,6),\ldots,(5,n),\ldots,(n-2,n-1),(n-2,n),(n-1,n)\}$ and $\mathcal{O}_{12}$ is any ordering of the set $\mathcal{S}_{12}=\{(1,5),\ldots,(1,n),\ldots,(4,5),\ldots,(4,n)\}$.

Obviously, the whole sweep on $A(\epsilon)$ reduces to the sweep on $A_{11}^{[0]}$ under the strategy $I_{\mathcal{O}_{1}}$ since all other Jacobi angles are zero. ∎

Let us show that the blocks $A_{12}^{[0]}$ and $A_{22}^{[0]}$ of the matrix $A^{(0)}=A(\epsilon)$ can be chosen such that all their entries are nonzero. Indeed, we can make other sets of the assumptions on $A(\epsilon)$. One such set of the assumptions is the following.

• (i)

$A_{11}^{[0]}$ is of order $4$ and such that

[TABLE]

holds when a full cycle of the Jacobi method under the strategy $I_{1}$ is applied to $A_{11}^{[0]}$. The existence of such an $A_{11}^{[0]}$ follows from (5.1) because we have proved the theorem for $n=4$.

• (ii)

We have $\|A_{12}^{[0]}\|_{F}\leq\epsilon^{\nu}S(A_{11}^{[0]})$, where $\nu$ satisfies $n\epsilon^{\nu-2}<1$.

• (iii)

We have $S(A_{22}^{[0]})\leq\epsilon^{2}S(A_{11}^{[0]})$ and

[TABLE]

where

[TABLE]

The pivot strategy $I$ is defined as in the proof of Theorem 5.3.

To keep the paper shorter we do not give a rigorous proof, but we make few essential remarks. After completing the sweep on $A_{22}^{[6]}$ the inequality (5.2) still holds, only the superscript $(6)$ on the left-hand side has to be replaced by $(M)$, $M=6+(n-4)(n-\nobreak 5)/2$. We also have $S(A_{22}^{(M)})\leq\epsilon^{2}S(A_{11}^{[0]})$ and $\|A_{12}^{(M)}\|_{F}=\|A_{12}^{[0]}\|_{F}\leq\epsilon^{\nu}S(A_{11}^{[0]})$. Due to the condition (5.3) all later angles will be bounded by some multiples of $\epsilon^{\nu}$. The sum of squares of the last $N-M$ pivot elements will be bounded by some multiple of $n\epsilon^{2\nu}S^{2}(A_{11}^{[0]})<\epsilon^{4}S^{2}(A_{11}^{[0]})$, which will eventually yield the required result.

Acknowledgements

The authors are thankful to the anonymous referees for their excellent remarks which improved the readability of the paper.

Appendix A Proofs related to Section 3

A.1. Proof of Proposition 3.1

We prove the proposition for $P=P_{12}$. The proof for the case $P=P_{34}$ is similar. Since the both Jacobi processes are cyclic, it is sufficient to prove the proposition for $r=1,2,3$. Let

[TABLE]

where

[TABLE]

Note that $P=P^{T}$. Let us inspect the product $P\mathsf{U}=P^{T}\mathsf{U}$. We have

[TABLE]

It remains to show that $\theta^{(0)}=\phi^{(1)}$, $\theta^{(1)}=\phi^{(0)}$, $\theta^{(2)}=\phi^{(3)}$, $\theta^{(3)}=\phi^{(2)}$, $\theta^{(4)}=-\phi^{(4)}$, $\theta^{(5)}=\phi^{(5)}$. Since

[TABLE]

it immediately follows from (1.1) that $\theta^{(0)}=\phi^{(1)}$, $\theta^{(1)}=\phi^{(0)}$. Thus, after completing the first two steps in each of the two processes, we have

[TABLE]

This shows that the relation (A.1) holds if $A^{(0)}$ and $\mathsf{A}^{(0)}$ are replaced by $A^{(2)}$ and $\mathsf{A}^{(2)}=P^{T}A^{(2)}P$, respectively. Checking the angle formula (1.1) we find that $\theta^{(2)}=\phi^{(3)}$, $\theta^{(3)}=\phi^{(2)}$ and therefore $\mathsf{A}^{(4)}=P^{T}A^{(4)}P$. The last check is the easiest one since the denominators in (1.1) for the angles $\theta^{(4)}$ and $\theta^{(5)}$ are opposite to those for the angles $\phi^{(4)}$ and $\phi^{(5)}$. ∎

A.2. Proof of Proposition 3.3

Let us denote $B^{(k)}=\mathcal{T}^{k}(A)$, $k\geq 0$. An easy calculation shows that

[TABLE]

Hence $B^{(6)}=\mathcal{T}^{6}(A)=(Q^{T})^{6}A^{(12)}Q^{6}=A^{(12)}$ and it is sufficient to show that the relation (3.8) holds for $0\leq k\leq 6$. We shall show

[TABLE]

Consider two processes, the first one is defined by the relation $B^{(k)}=\mathcal{T}^{k}(A)$, $k\geq 0$, and the second one is the Jacobi method under the strategy $I_{1}$. These two processes generate the matrices $B^{(k)}=(b_{rs}^{(k)})$, $k\geq 0$, and $A^{(l)}=(a_{rs}^{(l)})$, $l\geq 0$, respectively. The rotation angles at the step $k$ of the first process will be denoted by $\phi_{k}$ and $\psi_{k}$, $k\geq 1$. The rotation angle at the step $l$ of the Jacobi method will be denoted by $\varphi^{(l)}$, $l\geq 0$. Thus, $\phi_{k}$ and $\psi_{k}$ are used to compute $B^{(k)}$, while $\varphi^{(l-1)}$ is used to compute $A^{(l)}$.

For $k=0$ the assertion (3.8) takes the form $B^{(0)}=A^{(0)}$ which is correct since $B^{(0)}=A$ and $A^{(0)}=A$.

Let $k=1$. Then

[TABLE]

By Definition 3.2 angles $\phi_{1}$ and $\psi_{1}$ are the Jacobi angles which annihilate the elements of $A$ at positions $(1,3)$ and $(2,4)$. Therefore, we have $\phi_{1}=\varphi^{(0)}$ and $\psi_{1}=\varphi^{(1)}$, and consequently $A^{(2)}=R(2,4,\psi_{1})^{T}R(1,3,\phi_{1})^{T}AR(1,3,\phi_{1})R(2,4,\psi_{1})$. Thus, $B^{(1)}=Q^{T}A^{(2)}Q$, which had to be proved.

Let $k=2$. We use the fact that the assertion (3.8) holds for $k=1$. Using the relations (A.2) and (3.5) one obtains

[TABLE]

For the rotation angles which annihilate the elements at positions $(1,3)$ and $(2,4)$ we have

[TABLE]

hence,

[TABLE]

The relation $B^{(2)}=(Q^{2})^{T}A^{(4)}Q^{2}$ will hold provided that

[TABLE]

and the relation (A.5) will hold provided that

[TABLE]

It is easy to see that the relations (A.6) and (A.7) follow from the relations (3.5) and (A.4).

The proof for $k=3,4,5,6$ proceeds in the same manner as for $k=2$, but with different indices. ∎

A.3. Proof of Lemma 3.5

We shall use the notation from the proof of Proposition 3.3.

• (i)

If $a_{14}=0$ and $a_{23}=0$, then $S^{2}(A)=a_{13}^{2}+a_{24}^{2}$ and

[TABLE]

• (ii)

Since the first two pivot elements $a_{13}$ and $a_{24}$ are zero, the corresponding rotation angles $\varphi^{(0)}$ and $\varphi^{(1)}$ are zero as well, and $A^{(2)}=A$. The next two pivot elements $a_{14}$ and $a_{23}$ are the only possibly nonzero off-diagonal elements. Hence, $S^{2}(A^{(4)})=S^{2}(A)-a_{14}^{2}+a_{23}^{2}=0$.

• (iii)

Since $a_{13}=0$, we have $\phi_{1}=0$. The relations (3.6) imply

[TABLE]

We used the assumption $\psi_{1}\in[-\frac{\pi}{4},\frac{\pi}{4}]$. Using the relation (3.9), we have

[TABLE]

• (iv)

The proof is same as (iii), only $a_{13}$ and $\psi_{1}$ are used instead of $a_{24}$ and $\phi_{1}$.

• (v)

Let $a_{14}=0$. From the relations (3.6) we get

[TABLE]

We bounded the expression $\cos^{2}\phi_{1}\cos^{2}\psi_{1}+\sin^{2}\phi_{1}\sin^{2}\psi_{1}$ using the function $f(x,y)=1-(x^{2}+y^{2})+2x^{2}y^{2}$ for $x=\sin\phi_{1}$, $y=\sin\psi_{1}$ on $[-\frac{\sqrt{2}}{2},\frac{\sqrt{2}}{2}]\times[-\frac{\sqrt{2}}{2},\frac{\sqrt{2}}{2}]$. The minimum of that function equals $\frac{1}{2}$. Hence,

[TABLE]

• (vi)

The proof is same as $(v)$, only $a_{14}$ is used instead of $a_{23}$. ∎

Appendix B Proofs related to Section 4

B.1. Proof of Lemma 4.2

First, note that the following two inequalities hold:

[TABLE]

• (i)

Relations (4.15) and (B.1) imply

[TABLE]

The assertion follows from the fact that the rotation angles are from the interval $\displaystyle[-\frac{\pi}{4},\frac{\pi}{4}]$.

• (ii)

The assertion follows from (i).

• (iii)

Using the relations (B.2) and (B.1) we have

[TABLE]

• (iv)

From the relations (B.2) and (B.1) we have either

[TABLE]

or

[TABLE]

Hence, using (4.6) one obtains the lower bound for the tangents. The upper bound is obvious since the angles lie in the segment $[-\pi/4,\pi/4]$. For the latter assertion note that $x+x^{-1}\geq 2$ holds for any real $x\neq 0$ and equality is attained only for $x=1$. Recall that we can write $t=1-\gamma$, $0\leq\gamma\leq 4.444\delta_{k}$. Hence,

[TABLE]

• (v)

Specifying $t=\min\{|\tan\phi_{k}|,|\tan\psi_{k}|\}$ we have

[TABLE]

Since $\cot 2\phi_{k}$ and $\cot 2\psi_{k}$ have the opposite sign, the absolute value of their sum cannot be larger than the larger term.

• (vi)

Let $\eta_{k}=\cot 2\phi_{k}+\tan\phi_{k}+\cot 2\psi_{k}+\tan\psi_{k}$. Using the notation and ideas from the proof of (iv), we have

[TABLE]

This implies

[TABLE]

• (vii)

The proof is similar to the proof of (vii). If

[TABLE]

∎

B.2. Proof of Lemma 4.3

In terms of the elements of matrices $B^{(k)}$ and $B^{(k-1)}$ for $k\geq 1$ the angle formulas (3.7) take the form

[TABLE]

The relation (4.17) and Lemma 4.1(a) imply

[TABLE]

From the relations (3.6) for $k\geq 1$ we have

[TABLE]

Combining that with the angle formulas (B.3) one obtains

[TABLE]

Recall that for any $\zeta\in[-\frac{\pi}{4},\frac{\pi}{4}]\backslash\{0\}$ we have $\displaystyle|\tan\zeta+\cot 2\zeta|=\frac{1}{2}\left|\tan\zeta+\frac{1}{\tan\zeta}\right|\geq 1$. This implies

[TABLE]

The relations (B.5), (B.6) and (B.7) imply

[TABLE]

After squaring and summing the inequalities (B.8) and (B.9), using (4.4) and the inequality $(x+y)^{2}\leq 1.5x^{2}+3y^{2}$ which holds for any real $x$ and $y$, for $k\geq 1$ we get

[TABLE]

Bounding the term $|b_{22}^{(k)}-b_{44}^{(k)}|^{2}+|b_{11}^{(k)}-b_{33}^{(k)}|^{2}$ is simple. Using (B.3), (4.4) and Lemma 4.2(v) we obtain

[TABLE]

The relations (B.10) and (B.11) imply

[TABLE]

Bounding $|b_{11}^{(k)}-b_{44}^{(k)}|^{2}$ is more demanding. From the relations (3.6) for $k\geq 1$ we have

[TABLE]

From the relations (3.6) we also get

[TABLE]

Using (B.13), (B.14), (B.3), Lemma 4.2(iii), (4.4), Lemma 4.2(v), (4.16), (4.17) and (B.4) for $1\leq k\leq 3$ one obtains

[TABLE]

Here, for $k=1$ the term $2.222^{k-1}\delta_{k-1}\cdots\delta_{1}$ is replaced by one. Next, we use the inequality $\sqrt{a^{2}+b^{2}}\leq|a|+|b|$ and combine (B.12) and (B.15). After canceling by $S(B)$ we have

[TABLE]

Specifically, for $k=1,2,3$, one obtains

[TABLE]

Since

[TABLE]

it follows

[TABLE]

We used the fact that $\delta_{k}>0$ for $0\leq k\leq 4$. Combining the bounds for $\delta_{0}$ and $\delta_{1}$ in order to eliminate the term $1.225\delta_{1}$ we obtain

[TABLE]

To bound $\delta_{2}$ by $\delta_{4}$ we just insert the upper bound $0.0045$ for $\delta_{3}$ and $\delta_{4}$ into the expression within the parentheses. In a similar way (as $(9.35+7.778)\cdot 2\cdot 10^{-5}$) the bound for $\delta_{2}$ can be obtained. To bound $\delta_{1}$ by $\delta_{3}$ we use the bounds $0.000347$ and $0.0045$ for $\delta_{2}$ and $\delta_{4}$. Finally, the upper bounds for $\delta_{1}$ and $\delta_{0}$ are obtained (in this order) by inserting the best available bounds into the appropriate expressions. ∎

B.3. Proof of Lemma 4.4

• (i)

The assertion follows from (B.14), (4.16) and (4.17) or (B.4).

• (ii)

We use the parallelogram law, $(u+v)^{2}+(u-v)^{2}=2(u^{2}+v^{2})$ for $u,v$ real, and the definition of $\delta_{k}$ from (4.4). For $k=3,4$ the inequalities (4.20) follow from (4.19) and Lemma 4.3. We have

[TABLE]

• (iii)

We use the first case in Lemma 4.2(i)

[TABLE]

with $\delta_{k}^{2}<2\epsilon=2\cdot 10^{-5}$. The proof of (B.16) is the same if we use $\alpha_{k}^{\prime}$, $\beta_{k}^{\prime}$ instead of $\alpha_{k}$, $\beta_{k}$, respectively. From the relations (3.6) it follows that

[TABLE]

Hence, for $3\leq k\leq 4$ we can use the assertion (ii) to obtain

[TABLE]

Combining that with (B.16) one obtains $|\cot 2\phi_{k}-\cot 2\psi_{k}|>1.999894\delta_{k}$, $3\leq k\leq 4$. ∎

B.4. Proof of Lemma 4.3 in the case $|b_{14}-b_{23}|\leq\sqrt{2}\delta_{1}S(B)$

The relations (B.3)–(B.12) remain the same except for the relation (B.4) in which $\nu_{k}$ and $\nu$ are replaced by $\nu_{k}^{-}$ and $\nu^{-}=\nu_{0}^{-}$, respectively. For $k\geq 1$ the relation (B.13) can be written as

[TABLE]

and instead of (B.14) we use (B.17). The relation (B.15) takes the form

[TABLE]

for $1\leq k\leq 3$. Here we used the relations (B.18), (B.17), (4.27) and the assertions (iii) and (v) of Lemma 4.5. The rest of the proof follows the remaining lines in the proof of Lemma 4.3. ∎

B.5. Proof of Lemma 4.6

The proof of the first two assertions is quite similar to the proof of the appropriate assertions of Lemma 4.4. To prove the third assertion instead of the relation (B.16) we now have

[TABLE]

where we used $\delta_{k}^{2}<2\epsilon=2\cdot 10^{-5}$. The proof of (B.16) is the same if we use $\alpha_{k}^{\prime}$, $\beta_{k}^{\prime}$ instead of $\alpha_{k}$, $\beta_{k}$, respectively. Using (B.14) and the assertion (ii) for $3\leq k\leq 4$ we obtain

[TABLE]

Combining that with (B.19) we get $|\cot 2\phi_{k}+\cot 2\psi_{k}|>1.999894\delta_{k}$, $3\leq k\leq 4$. ∎

Appendix C Proofs related to Section 5

C.1. Proof of Proposition 5.2

We use the operator $\mathcal{T}$ from Definition 3.2. Let $B=B^{0}=H(\epsilon)$ and $B^{(k)}=\mathcal{T}^{k}(H)$ for $k\geq 1$.

For $k=1$ we compute $B^{(1)}$ from $B$. The elements $b_{13}$ and $b_{24}$ are annihilated and the off-norm reduction equals

[TABLE]

For the rotation angels $\phi_{1}$ and $\psi_{1}$ we have

[TABLE]

Using the notation from Lemma 4.2 we have

[TABLE]

Hence

[TABLE]

and

[TABLE]

Since $\displaystyle\arctan(x)=x-\frac{x^{3}}{3}+\frac{x^{5}}{5}-\frac{x^{7}}{7}+\cdots$, for $0<x\leq 1$ we have

[TABLE]

The relation (C.5) implies

[TABLE]

and consequently

[TABLE]

Since $\displaystyle\phi_{1}-\psi_{1}=\frac{\pi}{2}-(\alpha_{1}+\beta_{1})$, we have $\cos(\phi_{1}-\psi_{1})=\sin(\alpha_{1}+\beta_{1})$. Therefore, using the relation (C.6) we obtain

[TABLE]

Thus,

[TABLE]

Note that

[TABLE]

hence we have to bound $(b_{13}^{(1)})^{2}+(b_{24}^{(1)})^{2}$. From the relations (3.6) it follows that

[TABLE]

Inserting (C.9) and (C.10) into (C.8) and using (C.7) we obtain

[TABLE]

Let $k=2$. We have

[TABLE]

From the relations (C.9), (C.10) and (C.7) we conclude that the pivot elements $b_{13}^{(1)}$ and $b_{24}^{(1)}$ are negative. Using the relations (C.2) and (C.7) we bound their moduli from below

[TABLE]

Moreover, using (3.6) or (B.13) and (C.3), (C.4) we obtain

[TABLE]

and

[TABLE]

Hence, we conclude that $\phi_{2}<0$, $\psi_{2}>0$. Like in Lemma 4.2(i), we set

[TABLE]

From the relations (C.12), (C.3) and (C.4) we obtain

[TABLE]

Then, for $\alpha_{2}^{\prime}$ and $\beta_{2}^{\prime}$ it holds

[TABLE]

Next we bound the off-norm reduction in the third parallel step which equals $(b_{13}^{(2)})^{2}+(b_{24}^{(2)})^{2}$. We use the relations (4.13), (4.5) and (4.11) to obtain

[TABLE]

Finally, from (C.1), (C.8), (C.11) and (C.13) it follows

[TABLE]