Jacobi method for symmetric $4\times4$ matrices converges for every cyclic pivot strategy

TL;DR

This paper proves that the Jacobi method for symmetric 4x4 matrices converges globally for all 720 cyclic pivot strategies, with a convergence rate independent of the specific matrix or strategy used.

Contribution

It establishes the universal convergence of the Jacobi method for all cyclic strategies on 4x4 symmetric matrices, including the rate of convergence.

Findings

Convergence holds for all 720 cyclic strategies.

The off-diagonal norm decreases by a factor less than 1 after three cycles.

The result applies directly to the J-Jacobi method.

Abstract

The paper studies the global convergence of the Jacobi method for symmetric matrices of size $4$. We prove global convergence for all $720$ cyclic pivot strategies. Precisely, we show that inequality $S(A^{[t+3]})\leq\gamma S(A^{[t]})$, $t\geq1$, holds with the constant $\gamma<1$ that depends neither on the matrix $A$ nor on the pivot strategy. Here $A^{[t]}$ stands for the matrix obtained from $A$ after $t$ full cycles of the Jacobi method and $S(A)$ is the off-diagonal norm of $A$. We show why three consecutive cycles have to be considered. The result has a direct application on the $J$-Jacobi method.

Equations

$$A^{(k+1)}=R_k^T{A}^{(k)}{R}_k,k\ge 0,A^{(0)}=A=A^T,$$

$$S(X)=\frac{\sqrt{2}}{2}\Vert X-\text{diag}(X){\Vert }_F=\sqrt{\sum _{i=1}^{n-1}\sum _{j=i+1}^n{x}_{ij}^2},$$

$$\tan (2{\varphi }_k)=\frac{2a_{ij}^{(k)}}{a_{ii}^{(k)}-a_{jj}^{(k)}},-\frac{\pi }{4}\le {\varphi }_k\le \frac{\pi }{4},k\ge 0,$$

$$S^2(A^{(k+1)})=S^2(A^{(k)})-(a_{ij}^{(k)}{)}^2,k\ge 0.$$

$$S(A^{((t+\tau )N)})\le \gamma S(A^{(tN)}),t\ge t_0,N=\frac{n(n-1)n}{2}$$

$${\mathcal{O}}_I=I(0),I(1),\dots ,I(N-1).$$

$$I_{\mathcal{O}}(k)=(i_{\tau (k)},j_{\tau (k)}),0\le \tau (k)\le N-1,k\equiv \tau (k)(\mathrm{mod}N),k\ge 0.$$

$$\mathcal{O}^{\leftarrow}=(i_{N-1},j_{N-1}),\ldots,(i_{1},j_{1}),(i_{0},j_{0})\ \in\mathcal{\mbox{\Large$\mathcal{O}$}}(\mathbf{P}_{n})$$

$${\mathcal{O}}^{\prime }$$

$$\displaystyle\quad\ldots,\big{(}\min\{\mathsf{q}(i_{N-1}),\mathsf{q}(j_{N-1})\},\max\{\mathsf{q}(i_{N-1}),\mathsf{q}(j_{N-1})\}\big{)}.$$

$$\mathbf{P}_{4}=\big{\{}(1,2),(1,3),(1,4),(2,3),(2,4),(3,4)\big{\}}\quad\text{and}\quad N=6.$$

$$\mathcal{C}_{0}=\mathcal{C}_{0}(\mathbf{P}_{4}):=\big{\{}\mathcal{O}\in\mbox{\Large$\mathcal{O}$}(\mathbf{P}_{4})\ \big{|}\ I_{\mathcal{O}}(0)=(1,2)\big{\}}.$$

$$S(A^{[t+3]})\le \gamma S(A^{[t]}),t\ge 1.$$

$$\displaystyle\mathcal{C}_{c}^{(n)}=\big{\{}\mathcal{O}\in\mathcal{\mbox{\Large$\mathcal{O}$}}(\mathbf{P}_{n})\ \big{|}\$$

$$\displaystyle\pi_{j}\in\Pi^{(1,j-1)},\ 3\leq j\leq n\big{\}}$$

$$\displaystyle\mathcal{C}_{r}^{(n)}=\big{\{}\mathcal{O}\in\mathcal{\mbox{\Large$\mathcal{O}$}}(\mathbf{P}_{n})\ \big{|}\$$

$$\displaystyle\tau_{i}\in\Pi^{(i+1,n)},\ 1\leq i\leq n-2\big{\}}$$

$$\overleftarrow{\mathcal{C}_{c}}^{(n)}=\big{\{}\mathcal{O}\in\mathcal{\mbox{\Large$\mathcal{O}$}}(\mathbf{P}_{n})\ \big{|}\ \mathcal{O}^{\leftarrow}\in\mathcal{C}_{c}^{(n)}\big{\}},\quad\overleftarrow{\mathcal{C}_{r}}^{(n)}=\big{\{}\mathcal{O}\in\mathcal{\mbox{\Large$\mathcal{O}$}}(\mathbf{P}_{n})\ \big{|}\ \mathcal{O}^{\leftarrow}\in\mathcal{C}_{r}^{(n)}\big{\}},$$

$${\mathcal{C}}_{sp}^{(n)}={\mathcal{C}}_c^{(n)}\cup {\overleftarrow{{\mathcal{C}}_c}}^{(n)}\cup {\mathcal{C}}_r^{(n)}\cup {\overleftarrow{{\mathcal{C}}_r}}^{(n)},$$

$$S^2(A^{\prime })\le {\eta }_4{S}^2(A),{\eta }_4=\frac{27}{28}.$$

$$\eta_{n}=\max\Big{\{}1-2^{1-n},\ 1-\frac{2^{2-n}(1-\eta_{n-1})}{2^{2-n}+(n-2)\eta_{n-1}}\Big{\}}.$$

$$\eta_{4}=\max\Big{\{}\frac{7}{8},\frac{27}{28}\Big{\}}=\frac{27}{28}.$$

$$\mathcal{C}_{c}^{(4)}=\big{\{}\mathcal{O}\ \big{|}\ \mathcal{O}=(1,2),(\pi_{3}(1),3),(\pi_{3}(2),3),(\pi_{4}(1),4),(\pi_{4}(2),4),(\pi_{4}(3),4)\big{\}},$$

$$\pi_{3}={\small\left(\begin{array}[]{cc}1&2\\ \pi_{3}(1)&\pi_{3}(2)\\ \end{array}\right)},\quad\pi_{4}={\small\left(\begin{array}[]{ccc}1&2&3\\ \pi_{4}(1)&\pi_{4}(2)&\pi_{4}(3)\\ \end{array}\right)},$$

$$\mathcal{C}_{r}^{(4)}=\big{\{}\mathcal{O}\ \big{|}\ \mathcal{O}=(3,4),(2,\tau_{2}(3)),(2,\tau_{2}(4)),(1,\tau_{1}(2)),(1,\tau_{1}(3)),(1,\tau_{1}(4))\big{\}},$$

$$\tau_{2}={\small\left(\begin{array}[]{cc}3&4\\ \tau_{2}(3)&\tau_{2}(4)\\ \end{array}\right)},\quad\tau_{1}={\small\left(\begin{array}[]{ccc}2&3&4\\ \tau_{1}(2)&\tau_{1}(3)&\tau_{1}(4)\\ \end{array}\right)},$$

$${\overleftarrow{{\mathcal{C}}_c}}^{(4)}$$

$${\overleftarrow{{\mathcal{C}}_r}}^{(4)}$$

$${\mathcal{O}}_1$$

$${\mathcal{O}}_2$$

$${\mathcal{O}}_3$$

$${\mathcal{O}}_4$$

$${\mathcal{O}}_5$$

$${\mathcal{O}}_6$$

$${\mathcal{O}}_{13}$$

Full text

Erna Begović Kovač, Faculty of Chemical Engineering and Technology, University of Zagreb, Marulićev trg 19, 10000 Zagreb, Croatia, ebegovic@fkit.hr

Vjeran Hari, Department of Mathematics, University of Zagreb, Bijenička 30, 10000 Zagreb, Croatia, hari@math.hr

This work has been fully supported by Croatian Science Foundation under the project 3670.

1. Introduction

This paper considers the global convergence of the Jacobi method and is a continuation of the work from [2, 7]. It answers the question whether all cyclic Jacobi methods are convergent, at least for the case $n=4$. Furthermore, the result implies that any cyclic $J$-Jacobi method (see [15, 9]) for solving the generalized eigenvalue problem $Ax=\lambda Jx$ is globally convergent for $n=4$. The global convergence of the cyclic Jacobi methods for symmetric matrices has been considered in [4, 5, 10, 13, 14, 11] and lately in [1, 2, 6, 7]. These latest results have sufficiently enlarged the class of “convergent” strategies to make the general proof in the case $n=4$ possible.

Our results have a direct application on the corresponding block Jacobi methods with $2\times 2$ blocks. In that case pivot submatrices are $4\times 4$ matrices. In order to have a convergent block method we need a convergent core algorithm. Common approach in each block step of the block Jacobi method is to use the spectral decomposition of the pivot submatrix. This paper shows that, in the case of the blocks of size $2$, one can use the Jacobi algorithm with an arbitrary cyclic pivot strategy as the core algorithm. This choice of the core algorithm is natural because of the quadratic convergence and high relative accuracy of the Jacobi algorithm. In addition, it ensures the global convergence of the block Jacobi method (see [7], Theroem 2.10(ii)).

The eigenvalue problems for symmetric and $J$-symmetric matrices of size $4$ can be solved directly, by forming the quartic characteristic polynomial and using Cardano’s rootfinding formulas. However, efficiency and stability of such approach can be questioned, since there are fast and stable algorithms based on Jacobi and $J$-Jacobi methods. Especially in the context of block Jacobi methods with $2\times 2$ blocks when the pivot submatrices are of size $4$ and most of the time they are nearly diagonal. On such matrices Jacobi methods are very efficient (just few sweeps are needed) and highly accurate [12].

Jacobi method for symmetric matrices is an iterative process of the form

[TABLE]

where $R_{k}=R(i(k),j(k),\phi_{k})$, $k\geq 0$, are plane rotations. Here $R(p,q,\psi)$, $p<q$, stands for the rotation in the $(p,q)$ plane, whose $(p,q)$-element is equal to $-\sin\psi$. The goal of the $k$-th step is to reduce the off-diagonal norm of the current matrix $A^{(k)}$. The off-diagonal norm, or shortly off-norm, of a symmetric $n\times n$ matrix $X=(x_{rs})$ is defined by

[TABLE]

where $\|X\|_{F}$ denotes the Frobenius norm of $X$. The rotation angle $\phi_{k}$ is defined by

[TABLE]

and this choice results in the annihilation of the pivot element $a_{ij}^{(k)}$ of $A^{(k)}$. Throughout this paper it is presumed that the Jacobi method is defined by the rotation angles from the interval $[-\frac{\pi}{4},\frac{\pi}{4}]$. The pair $(i,j)$, $i<j$, depending on $k$, is called pivot pair. Hence $i=i(k)$, $j=j(k)$, and letters $i$ and $j$ will be reserved for pivot indices. A method is said to be globally convergent if, for any initial $A$, the sequence of matrices $(A^{(k)},k\geq 0)$ generated by the iterative process (1.1) converges to some diagonal matrix.

The diagonal elements of the iteration $A^{(k)}$ generated by the Jacobi method always converge [11]. Hence, a method is globally convergent if and only if the sequence of the off-norms $(S(A^{(k)}),k\geq 0)$ converges to zero for any initial $A$. This sequence is nonincreasing because

[TABLE]

Therefore, it is sufficient to find a subsequence of $(S(A^{(k)}),k\geq 0)$ that converges to zero. In particular, it is sufficient to prove that

[TABLE]

holds for every cyclic strategy and every initial $A$, with $\tau,t_{0}\in\mathbb{N}$ and the constant $\gamma<1$, that depends neither on $A$ nor on the pivot strategy. In the case $n=4$, we shall prove that relation (1.2) holds with $\tau=3$, $t_{0}=1$ and $\gamma=1-10^{-5}$.

For a symmetric $4\times 4$ matrix there are altogether $720$ cyclic pivot strategies. The convergence properties of the Jacobi method greatly vary for different pivot strategies [2]. Say, for some parallel strategies from Section 3.3 and some initial symmetric matrices, the quotient $S(A^{(N)})/S(A)$ can be arbitrary close to $1$ [2, Proposition 5.2], and in the second cycle that quotient can be as small as $10^{-26}$ [2, Example 5.1]. Hence, for convergence considerations one has to take into account more than one cycle.

The paper is divided into $5$ sections. In Section 2 we present a brief theory of pivot strategies and pivot orderings. The main result of the paper is formulated in Theorem 3. In Section 3 different classes of cyclic pivot strategies are analyzed. Specifically, the generalized serial pivot strategies from [1, 7] are studied in Subsections 3.1 and 3.2 and the parallel strategies from [1, 2] in Subsection 3.3. In Section 4 the proof of the main theorem is presented. Finally, in Section 5 we show how the new results can be applied to the $J$-Jacobi method.

Most of the results presented here are a part of the unpublished thesis [1].

2. Cyclic pivot strategies and pivot orderings

In the $k$-th step of the Jacobi method, the pivot pair $(i(k),j(k))$ is selected and the pivot element $a_{ij}^{(k)}$ of $A^{(k)}$ is annihilated. The rule for choosing the pivot pairs is called the pivot strategy. Pivot strategy can be identified with a function $I:\mathbb{N}_{0}\rightarrow\mathbf{P}_{n}$, where $\mathbb{N}_{0}=\{0,1,2,3,\ldots\}$, \mathbf{P}_{n}=\big{\{}(r,s)\ \big{|}\ 1\leq r<s\leq n\big{\}}, and $n$ is the size of the matrix. If there is $T\in\mathbb{N}$ such that $I(k+T)=I(k)$, $k\geq 0$, then $I$ is periodic with period $T$. In addition, if $T=N\equiv\frac{n(n-1)}{2}$ and \{I(k)\ \big{|}\ 0\leq k\leq T-1\}=\mathbf{P}_{n}, the pivot strategy is cyclic. The pivot pair $I(k)$ will be briefly denoted by $(i,j)$ when $k$ is implied.

Let \mbox{\Large\mathcal{O}}(\mathbf{P}_{n}) denote the set of all orderings of $\mathbf{P}_{n}$. For a given pivot strategy $I$, the corresponding pivot ordering \mathcal{O}_{I}\in\mbox{\Large\mathcal{O}}(\mathbf{P}_{n}) is defined as

[TABLE]

It corresponds to one cycle or sweep of the method. On the other hand, for a given ordering $\mathcal{O}=(i_{0},j_{0}),(i_{1},j_{1}),\ldots,(i_{N-1},j_{N-1})$ from \mathcal{\mbox{\Large\mathcal{O}}}(\mathbf{P}_{n}), the corresponding pivot strategy $I_{\mathcal{O}}$ is defined by

[TABLE]

Hence, $\mathcal{O}_{I_{\mathcal{O}}}=\mathcal{O}$ and we can use the terms “ordering” and “pivot ordering” as synonyms.

We say that the ordering \mathcal{O}\in\mathcal{\mbox{\Large\mathcal{O}}}(\mathbf{P}_{n}) and the pivot strategy $I_{\mathcal{O}}$ are convergent if relation (1.2) holds for $I_{\mathcal{O}}$.

An admissible transposition on \mathcal{O}\in\mathcal{\mbox{\Large\mathcal{O}}}(\mathbf{P}_{n}) is a transposition of two adjacent terms $(i_{r},j_{r}),$ $(i_{r+1},j_{r+1})\rightarrow(i_{r+1},j_{r+1}),(i_{r},j_{r})$, $0\leq r<N-1,$ provided that $\{i_{r},j_{r}\}$ and $\{i_{r+1},j_{r+1}\}$ are disjoint. We say that such pairs $(i_{r},j_{r})$ and $(i_{r+1},j_{r+1})$ commute.

Two orderings \mathcal{O},\mathcal{O}^{\prime}\in\mathcal{\mbox{\Large\mathcal{O}}}(\mathbf{P}_{n}) are:

• (i)

equivalent (we write $\mathcal{O}\sim\mathcal{O}^{\prime}$) if one can be obtained from the other using a finite number of admissible transpositions.

• (ii)

shift-equivalent ($\mathcal{O}\stackrel{{\scriptstyle s}}{{\sim}}\mathcal{O}^{\prime}$) if $\mathcal{O}=[\mathcal{O}_{1},\mathcal{O}_{2}]$ and $\mathcal{O}^{\prime}=[\mathcal{O}_{2},\mathcal{O}_{1}]$, where $[\ ,\ ]$ stands for the concatenation. The number of pairs in the sequence $\mathcal{O}_{1}$ is called the length of the shift.

• (iii)

weakly equivalent ($\mathcal{O}\stackrel{{\scriptstyle w}}{{\sim}}\mathcal{O}^{\prime}$) if there are \mathcal{O}_{i}\in\mathcal{\mbox{\Large\mathcal{O}}}(\mathbf{P}_{n}), $1\leq i\leq r$, such that in the sequence $\mathcal{O},\mathcal{O}_{1},\ldots,\mathcal{O}_{r},\mathcal{O}^{\prime}$ every two adjacent terms are equivalent or shift-equivalent.

It is easy to check that $\sim$, $\stackrel{{\scriptstyle s}}{{\sim}}$ and $\stackrel{{\scriptstyle w}}{{\sim}}$ are equivalence relations on the set \mathcal{\mbox{\Large\mathcal{O}}}(\mathbf{P}_{n}). Two pivot strategies $I_{\mathcal{O}}$ and $I_{\mathcal{O}^{\prime}}$ are equivalent (shift-equivalent, weakly equivalent) if the related pivot orderings from relation (2.1) are equivalent (shift-equivalent, weakly equivalent).

We will need two more relations between pivot orderings. They have been introduced in [1, 7]. These relations will be used to prove the global convergence of the Jacobi method under different pivot orderings.

Definition 1**.**

Let \mathcal{O}\in\mathcal{\mbox{\Large\mathcal{O}}}(\mathbf{P}_{n}), $\mathcal{O}=(i_{0},j_{0}),(i_{1},j_{1}),\ldots,(i_{N-1},j_{N-1})$. Then

[TABLE]

is the reverse ordering to $\mathcal{O}$. The pivot strategy $I_{\mathcal{O}}^{\leftarrow}=I_{\mathcal{O}^{\leftarrow}}$ is reverse to $I_{\mathcal{O}}$.

Definition 2**.**

Let \mathcal{O}\in\mathcal{\mbox{\Large\mathcal{O}}}(\mathbf{P}_{n}), $\mathcal{O}=(i_{0},j_{0}),(i_{1},j_{1}),\ldots,(i_{N-1},j_{N-1})$. Then \mathcal{O}^{\prime}\in\mathcal{\mbox{\Large\mathcal{O}}}(\mathbf{P}_{n}) is permutation equivalent to $\mathcal{O}$ if there is a permutation $\mathsf{q}$ of the set $\{1,2,\ldots,n\}$ such that

[TABLE]

We write $\mathcal{O}^{\prime}=\mathcal{O}(\mathsf{q})$ or $\mathcal{O}\stackrel{{\scriptstyle\mathsf{q}}}{{\sim}}\mathcal{O}^{\prime}$. The general notation $\mathcal{O}^{\prime}\stackrel{{\scriptstyle p}}{{\sim}}\mathcal{O}$ will stand for permutation equivalence between $\mathcal{O}^{\prime}$ and $\mathcal{O}$ with some permutation.

Let us now consider the case $n=4$. We have

[TABLE]

Since the cardinality of the set $\mathbf{P}_{4}$ is $6$, there are $6!$ orderings of $\mathbf{P}_{n}$ and consequently $720$ cyclic pivot strategies. The ones of our special interest are those (cyclic) strategies that start with the pivot element at position $(1,2)$, i.e. for which $I(0)=(1,2)$. Set

[TABLE]

There are $5!=120$ pivot orderings in $\mathcal{C}_{0}$. The global convergence results for all strategies $I_{\mathcal{O}}$, \mathcal{O}\in\mbox{\Large\mathcal{O}}(\mathbf{P}_{4}), will follow from the results obtained for $I_{\mathcal{O}}$, $\mathcal{O}\in\mathcal{C}_{0}$.

The main result of the paper is given in the following theorem.

Theorem 3**.**

Let $A$ be a symmetric $4\times 4$ matrix and let \mathcal{O}\in\mbox{\Large\mathcal{O}}(\mathbf{P}_{4}). Let $A^{[t]}$ be obtained from $A$ by applying $t$ sweeps of the cyclic Jacobi method defined by the pivot strategy $I_{\mathcal{O}}$. Then there is a constant $\gamma$, $0\leq\gamma<1$, that depends neither on $A$ nor on the pivot strategy, such that

[TABLE]

Proof.

The proof uses results from Section 3, and is moved to Section 4. ∎

3. Auxiliary results

Special classes of cyclic pivot strategies have been introduced and studied in [1, 7]. We briefly recall notation and convergence results from [7]. The results refer to the Jacobi method for a symmetric $n\times n$ matrix. Later on, we will consider the case $n=4$.

3.1. Classes of pivot orderings $\mathcal{C}_{c}^{(4)}$ and $\mathcal{C}_{r}^{(4)}$

Let us recall definitions of the sets $\mathcal{C}_{c}^{(n)}$ and $\mathcal{C}_{r}^{(n)}$ from [7]. Let $\Pi^{(l_{1},l_{2})}$ denote the set of all permutations of the set $\{l_{1},l_{1}+1,\ldots\ ,l_{2}\}$, $l_{1}\leq l_{2}$. Then,

[TABLE]

is the set of column-wise orderings with permutations of \mathcal{\mbox{\Large\mathcal{O}}}(\mathbf{P}_{n}). The columns are taken from left to right. Similarly,

[TABLE]

is the set of row-wise orderings with permutations of \mathcal{\mbox{\Large\mathcal{O}}}(\mathbf{P}_{n}). Here, the rows are taken from bottom to top.

The sets

[TABLE]

contain the orderings reverse to those from $\overleftarrow{\mathcal{C}_{c}}^{(n)}$, $\overleftarrow{\mathcal{C}_{r}}^{(n)}$, respectively. This means that in the orderings from $\overleftarrow{\mathcal{C}_{c}}^{(n)}$ ($\overleftarrow{\mathcal{C}_{r}}^{(n)}$) the pairs are arranged column-by-column from right to left (row-by-row from top to bottom), with permutations inside the columns (rows).

The union of all these sets is denoted by $\mathcal{C}_{sp}^{(n)}$,

[TABLE]

and is called the set of serial orderings with permutations of \mathcal{\mbox{\Large\mathcal{O}}}(\mathbf{P}_{n}).

The convergence result for the Jacobi method defined by the pivot strategies $I_{\mathcal{O}}$, $\mathcal{O}\in\mathcal{C}_{sp}^{(4)}$, is given in Proposition 4. This is a special case of [7, Corollary 3.3] for $n=4$.

Proposition 4**.**

Let $A$ be a symmetric $4\times 4$ matrix and $\mathcal{O}\in\mathcal{C}_{sp}^{(4)}$. Let $A^{\prime}$ be obtained from $A$ by applying one sweep of the cyclic Jacobi method defined by the strategy $I_{\mathcal{O}}$. Then

[TABLE]

Proof.

It has been shown in [7, Corollary 3.3] that the relation (3.3) holds for a symmetric matrix $A$ of size $n$ and $\mathcal{O}\in\mathcal{C}_{sp}^{(n)}$ with the constant $\eta_{n}$ satisfying

[TABLE]

If $n=2$, matrix $A$ is diagonalized by only one step of the Jacobi method, which corresponds to one cycle. Therefore, $\eta_{2}=0$. Hence, from relation (3.4) we obtain

[TABLE]

∎

Thus, all orderings from $\mathcal{C}_{sp}^{(n)}$ are convergent. Let us see how the serial orderings with permutations look like for the case $n=4$.

The relations (3.1) and (3.2) take the form

[TABLE]

with

[TABLE]

and

[TABLE]

with

[TABLE]

respectively. For the classes of reverse orderings, from (3.1) we obtain

[TABLE]

Let us list the orderings from $\mathcal{C}_{0}\cap\mathcal{C}_{sp}^{(4)}$. First we list those from $\mathcal{C}_{0}\cap\mathcal{C}_{c}^{(4)}$:

[TABLE]

The orderings that belong to $\mathcal{C}_{0}\cap\overleftarrow{\mathcal{C}_{r}}^{(4)}$ are the following:

[TABLE]

The intersection of the sets $\mathcal{C}_{0}$ and $\mathcal{C}_{r}^{(4)}$ is empty because the orderings from $\mathcal{C}_{0}$ start with pivot pair $(1,2)$ and the orderings from $\mathcal{C}_{r}^{(4)}$ start with $(3,4)$. Also, $\mathcal{C}_{0}\cap\overleftarrow{\mathcal{C}_{c}}^{(4)}=\emptyset$ because the orderings from $\overleftarrow{\mathcal{C}_{c}}^{(4)}$ start with a pair from the $4$th column.

3.2. Generalized serial orderings and strategies

As has been proved in [7], the set of convergent pivot orderings $\mathcal{C}_{sp}^{(4)}$ can be further generalized to the set $\mathcal{C}_{sg}^{(4)}$ given by

[TABLE]

The elements of $\mathcal{C}_{sg}^{(4)}$ are called generalized serial orderings. Proposition 5 below, which is a special case of [7, Theorem 3.11], proves the global convergence of the cyclic Jacobi method defined by orderings from $\mathcal{C}_{sg}^{(4)}$. The constant $\gamma_{4}$ from relation (3.6) is the same as $\gamma_{4}$ in relation (3.3).

Proposition 5**.**

Let $A$ be a symmetric $4\times 4$ matrix and $\mathcal{O}\in\mathcal{C}_{sg}^{(4)}$. Suppose the chain connecting $\mathcal{O}$ with $\mathcal{O}^{\prime\prime}\in\mathcal{C}_{sp}^{(4)}$ in (3.5) contains $d$ relations of shift-equivalence. Let $A^{[d+1]}$ be obtained from $A$ by applying $d+1$ sweeps of the cyclic Jacobi method defined by the strategy $I_{\mathcal{O}}$. Then

[TABLE]

Proof.

The result follows from [7, Theorem 3.11] and Proposition 4. In [7, Theorem 3.11] one has to use the partition $\pi=(1,1,1,1)$. Proposition 4 is used only to calculate $\eta_{4}$. ∎

Let us first see how far the set $\mathcal{C}_{sp}^{(4)}$ can be extended without using the permutation equivalence. In the following relations, for the sake of compactness, we shall mark some orderings from \mbox{\Large\mathcal{O}}(\mathbf{P}_{4}) by $\cdot$. Moreover, the equivalence relation that transposes $r$-th and $s$-th pair in the ordering will be denoted $\stackrel{{\scriptstyle r-s}}{{\sim}}$, and the shift-equivalence with the shift of the length $l$, $\stackrel{{\scriptstyle s(l)}}{{\sim}}$. Finally, in this subsection, the pairs in pivot orderings will be denoted without parenthesis, say $(r,s)$ we will denoted by rs. We do this for the conciseness and in order to fit two orderings into one line.

The orderings from $\mathcal{C}_{0}$ equivalent to some orderings from $\mathcal{C}_{c}^{(4)}\cup\overleftarrow{\mathcal{C}_{r}}^{(4)}$ are the following:

[TABLE]

The orderings linked to those from $\mathcal{C}_{sp}^{(4)}$ with one shift-equivalence are as follows:

[TABLE]

Further on, we come to the orderings from $\mathcal{C}_{0}$ that are weakly equivalent to the ones from $\mathcal{C}_{sp}$. They are listed below:

[TABLE]

Using the relations $\sim$, $\stackrel{{\scriptstyle s}}{{\sim}}$ and $\stackrel{{\scriptstyle w}}{{\sim}}$, we have enlarged the set of $16$ orderings belonging to $\mathcal{C}_{sp}^{(4)}$ to the set of $64$ orderings belonging to $\mathcal{C}_{sg}^{(4)}$. To continue, we use the permutation equivalence relation. We will only need four different permutations. The first one is

[TABLE]

Applying $\mathsf{p}_{1}$ we obtain the following convergent orderings:

[TABLE]

Next, we use the permutation

[TABLE]

It leads to the following convergent orderings:

[TABLE]

We shall also use the permutation equivalence $\stackrel{{\scriptstyle p}}{{\sim}}$ with

[TABLE]

It yields the following convergent orderings:

[TABLE]

Finally, for one pivot ordering we use the permutation \mathsf{q}_{4}=\left(\begin{array}[]{cccc}1&2&3&4\\ 1&3&4&2\\ \end{array}\right). We have

[TABLE]

There are $16$ orderings left in \mathcal{C}_{0}\big{\backslash}\mathcal{C}_{sg}^{(4)} that cannot be linked to any of the orderings from $\mathcal{C}_{sg}^{(4)}$ by using the equivalence relations $\sim$, $\stackrel{{\scriptstyle s}}{{\sim}}$, $\stackrel{{\scriptstyle w}}{{\sim}}$ or $\stackrel{{\scriptstyle p}}{{\sim}}$. These are the parallel orderings.

3.3. Parallel strategies

Behavior of the Jacobi method under parallel pivot strategies has been studied in [2]. Parallel pivot strategies are interesting especially because, unlike the serial ones, they can force the Jacobi method to be very fast or very slow within one cycle. As a representative of parallel orderings for $n=4$ we take the pivot ordering

[TABLE]

Its “parallelism” manifests in the fact that pairs

[TABLE]

commute. Therefore, the corresponding Jacobi transformations commute and can be computed and applied independently. Instead of six sequential steps needed to complete one cycle, only three parallel steps are used.

If $\mathcal{O}\sim\mathcal{O}_{\text{par}}$, it is easy to see that the only allowed transpositions are those of the pairs within the curly braces from (3.8). Thus, after each parallel step, the Jacobi method defined by such $I_{\mathcal{O}}$ generates the same matrix as the Jacobi method defined by $I_{\mathcal{O}_{\text{par}}}$.

Another representative of parallel orderings is

[TABLE]

It is easily seen that $\mathcal{O}_{\text{par}}^{\prime}$ is not weakly equivalent to $\mathcal{O}_{\text{par}}$. However, these two orderings are permutation equivalent, in particular, we have

[TABLE]

If $\mathcal{O}\stackrel{{\scriptstyle p}}{{\sim}}\mathcal{O}_{\text{par}}$, then $\mathcal{O}$ results from $\mathcal{O}_{\text{par}}$ or $\mathcal{O}_{\text{par}}^{\prime}$ by applying an appropriate $\sim$ and afterwards an appropriate $\stackrel{{\scriptstyle s}}{{\sim}}$ with shift of length [math], $2$ or $4$, or vice versa (first $\stackrel{{\scriptstyle s}}{{\sim}}$ then $\sim$). The same is true for $\mathcal{O}\stackrel{{\scriptstyle p}}{{\sim}}\mathcal{O}_{\text{par}}^{\prime}$. This follows from the fact that permutations are bijections and commuting pairs are mapped into commuting pairs.

For the Jacobi method defined by ordering $\mathcal{O}_{\text{par}}$ from (3.7) or $\mathcal{O}_{\text{par}}^{\prime}$ from (3.9), Proposition 6 holds. Note that the condition $a_{12}=0$ and $a_{34}=0$ is not a restriction when we are considering the global convergence. If that condition does not hold for the original matrix $A$ it will hold at the beginning of each cycle of the Jacobi method, starting with the second one.

Proposition 6**.**

Let $A$ be a symmetric $4\times 4$ matrix such that $a_{12}=0$, $a_{34}=0$, and let $A^{[2]}$ be obtained by applying two sweeps of the Jacobi method under the strategy $I_{\mathcal{O}_{\text{par}}}$ ($I_{\mathcal{O}_{\text{par}}^{\prime}}$) to $A$. Then

[TABLE]

Estimate (3.10) holds if $\mathcal{O}_{\text{par}}$ ($\mathcal{O}_{\text{par}}^{\prime}$) is replaced by any $\mathcal{O}$ such that $\mathcal{O}\sim\mathcal{O}_{\text{par}}$ ($\mathcal{O}\sim\mathcal{O}_{\text{par}}^{\prime}$).

Proof.

The first assertion for $I_{\mathcal{O}_{\text{par}}}$ is [2, Theorem 3.4]. The second one for $\mathcal{O}\sim\mathcal{O}_{\text{par}}$ follows from the fact that the Jacobi methods defined by the strategies $I_{\mathcal{O}_{\text{par}}}$ and $I_{\mathcal{O}}\sim I_{\mathcal{O}_{\text{par}}}$ generate the same matrix $A^{[2]}$.

To prove the assertion for $I_{\mathcal{O}_{\text{par}}^{\prime}}$, let $\mathsf{P}=[e_{1},e_{2},e_{4},e_{3}]$, where $[e_{1},e_{2},e_{3},e_{4}]=I$. Let $(A^{(k)},k\geq 0)$ and $(\tilde{A}^{(k)},k\geq 0)$ be two sequences of matrices generated by the Jacobi method defined by the strategies $I_{\mathcal{O_{\text{par}}}}$ and $I_{\mathcal{O_{\text{par}}}^{\prime}}$, respectively. From [2, Proposition 3.1] we have

[TABLE]

Therefore, $S(\tilde{A}^{[0]})=S(A^{[0]})$ and $S(\tilde{A}^{[2]})=S(A^{[2]})$, which proves the relation (3.10) for $I_{\mathcal{O}_{\text{par}}^{\prime}}$. The final assertion for $\mathcal{O}\sim\mathcal{O}_{\text{par}}^{\prime}$ is obtained by the same argument as for $\mathcal{O}\sim\mathcal{O}_{\text{par}}$, but $I_{\mathcal{O}_{\text{par}}^{\prime}}$ is used instead of $I_{\mathcal{O}_{\text{par}}}$. ∎

The following corollary uses the same $\gamma_{4}^{\prime}$.

Corollary 7**.**

Let \mathcal{O},\mathcal{O}^{\prime},\mathcal{O}^{\prime\prime}\in\mbox{\Large\mathcal{O}}(\mathbf{P}_{4}) be such that either $\mathcal{O}\stackrel{{\scriptstyle s}}{{\sim}}\mathcal{O}^{\prime}\sim\mathcal{O}_{\text{par}}$ or $\mathcal{O}\sim\mathcal{O}^{\prime\prime}\stackrel{{\scriptstyle s}}{{\sim}}\mathcal{O}_{\text{par}}$. Let $A$ be a symmetric $4\times 4$ matrix and let $A^{[t]}$ be obtained after applying $t$, $t\geq 1$, sweeps of the Jacobi method under the strategy $I_{\mathcal{O}}$ to $A$. Then

[TABLE]

Moreover, if $t=0$, then the relation (3.11) holds if the length of the shift is not one or if $a_{12}=a_{34}=0$.

The same assertion holds provided that $\mathcal{O}_{\text{par}}$ is replaced by $\mathcal{O}_{\text{par}}^{\prime}$.

Proof.

First, consider the case $\mathcal{O}\stackrel{{\scriptstyle s}}{{\sim}}\mathcal{O}_{\text{par}}$. The sequence of pivot pairs $[\mathcal{O},\mathcal{O},\mathcal{O}]$ contains the sequence $[\mathcal{O}_{\text{par}},\mathcal{O}_{\text{par}}]$. Since $t\geq 1$, Proposition 6 can be applied to $A^{[t]}$ and assertion (3.11) follows.

If the length of shift is not one, then the sequence of the pairs $(1,2),(3,4)$ precedes the sequence $[\mathcal{O}_{\text{par}},\mathcal{O}_{\text{par}}]$. Elements at the positions $(1,2)$ and $(3,4)$ will both be zero prior to the application of Proposition 6. Thus, relation (3.11) holds for $t=0$ as well. If the length of shift is one, only the pair $(3,4)$ will precede the sequence $[\mathcal{O}_{\text{par}},\mathcal{O}_{\text{par}}]$. Therefore, in order to apply Proposition 6, we should require $a_{12}=0$.

If $\mathcal{O}\stackrel{{\scriptstyle s}}{{\sim}}\mathcal{O}^{\prime}\sim\mathcal{O}_{\text{par}}$, the reasoning is the same, where the role of $\mathcal{O}_{\text{par}}$ is now played by $\mathcal{O}^{\prime}$. Sequence $\mathcal{O}^{\prime}$ can differ from $\mathcal{O}_{\text{par}}$ in at most three transpositions of the pairs from (3.8). Therefore, if the length of the shift is one, one of the pairs $(1,2),(3,4)$ will precede the sequence $[\mathcal{O}_{\text{par}},\mathcal{O}_{\text{par}}]$ and we should require $a_{12}=a_{34}=0$.

If $\mathcal{O}\sim\mathcal{O}^{\prime\prime}\stackrel{{\scriptstyle s}}{{\sim}}\mathcal{O}_{\text{par}}$, the reasoning is similar, only the role of $\mathcal{O}$ is now played by $\mathcal{O}^{\prime\prime}$. Since $\mathcal{O}\sim\mathcal{O}^{\prime\prime}$ the sequence $\mathcal{O}$ can differ from $\mathcal{O}^{\prime\prime}$ in at most three transpositions of the pairs from (3.8). If the length of the shift is one, only one of the pairs $(1,2),(3,4)$ will precede the sequence $[\mathcal{O}_{\text{par}},\mathcal{O}_{\text{par}}]$ and we should require $a_{12}=a_{34}=0$.

The assertions which use $\mathcal{O}_{\text{par}}^{\prime}$ are proved in the same way. Every appearance of $\mathcal{O}_{\text{par}}$ is replaced by $\mathcal{O}_{\text{par}}^{\prime}$. ∎

There are $16$ orderings from $\mathcal{C}_{0}$ that can be fully parallelized. These orderings are exactly those from \mathcal{C}_{0}\big{\backslash}\mathcal{C}_{sg}^{(4)}. Using the equivalence and shift-equivalence relations (i.e. the weak equivalence), we can link the following eight orderings from $\mathcal{C}_{0}$ with $\mathcal{O}_{\text{par}}$:

[TABLE]

In the same manner the other eight orderings from $\mathcal{C}_{0}$ can be linked with $\mathcal{O}_{\text{par}}^{\prime}$:

[TABLE]

This shows that all $120$ orderings from $\mathcal{C}_{0}$ are convergent.

4. Proof of Theorem 3

To prove Theorem 3 we use the results from sections 2 and 3. We know that there are $720$ cyclic orderings in \mbox{\Large\mathcal{O}}(\mathbf{P}_{4}) and $120$ of them are in $\mathcal{C}_{0}$. Each ordering from \mbox{\Large\mathcal{O}}(\mathbf{P}_{4}) either belongs to $\mathcal{C}_{sg}^{(4)}$ or is weakly equivalent to either $\mathcal{O}_{par}$ or $\mathcal{O}_{par}^{\prime}$.

Let $A$ be an arbitrary symmetric $4\times 4$ matrix. By $A^{[t]}$ we denote the matrix obtained from $A$ after $t$ sweeps of the Jacobi method. Recall that $\gamma_{4}^{\prime}=1-10^{-5}$ and $\eta_{4}=\frac{27}{28}$. Set $\gamma_{4}=\sqrt{\eta_{4}}=\frac{3\sqrt{21}}{14}<1$.

First we deal with the pivot orderings $\mathcal{O}_{i}$, $1\leq i\leq 120$, from $\mathcal{C}_{0}$ that have been analyzed in Section 3. For $i\in\{1,\ldots,16\}$, we have $\mathcal{O}_{i}\in\mathcal{C}_{sp}^{(4)}$ and Proposition 4 implies

[TABLE]

For $i\in\{17,\ldots,104\}$, we use Proposition 5. If $i\in\{17,18,19,20,82,90\}$, then there are no shift-equivalences linking $\mathcal{O}_{i}$ to an ordering from $\mathcal{C}_{sp}^{(4)}$. Therefore, for the corresponding pivot strategies relation (4.1) holds. Otherwise, there is one shift-equivalence in the corresponding links and we have

[TABLE]

For $i\in\{105,\ldots,120\}$, the orderings $\mathcal{O}_{i}$ are linked to the ordering $\mathcal{O}_{par}$ from (3.7) or to $\mathcal{O}_{par}^{\prime}$ from (3.9) with one shift-equivalence. For $i\in\{105,110,$ $111,112\}$ the length of the shift is two and from Proposition 6 it follows

[TABLE]

For $i\in\{117,\ldots,120\}$ the length of the shift is also two and the relation(4.3) follows from the same proposition. If $i\in\{106,\ldots,109,113,\ldots,116\}$ the length of the shift is one, so we use Corollary 7 to obtain

[TABLE]

Assume now that \mathcal{O}\in\mbox{\Large\mathcal{O}}(\mathbf{P}_{4})\big{\backslash}\mathcal{C}_{0}. Then $\mathcal{O}$ is shift-equivalent to some $\mathcal{O}_{i}\in\mathcal{C}_{0}$. If $\mathcal{O}$ is shift-equivalent to $\mathcal{O}_{i}$, $1\leq i\leq 104$, then $\mathcal{O}$ can be linked to some $\mathcal{O}^{\prime\prime}\in\mathcal{C}_{sp}$ with at most two shift-equivalences. Hence, applying Proposition 5 we obtain

[TABLE]

Finally, let \mathcal{O}\in\mbox{\Large\mathcal{O}}(\mathbf{P}_{4})\big{\backslash}\mathcal{C}_{0} be shift-equivalent to some $\mathcal{O}_{i}\in\mathcal{C}_{0}$, $105\leq i\leq 120$. Then, one of the two possibilities holds.

• (i)

$\mathcal{O}\stackrel{{\scriptstyle s}}{{\sim}}\mathcal{O}_{i}$, $i\in\{105,\ldots,112\}$, and there is \mathcal{O}^{\prime}\in\mbox{\Large\mathcal{O}}(\mathbf{P}_{4}) such that

[TABLE]

By the transitivity property of $\stackrel{{\scriptstyle s}}{{\sim}}$ we have $\mathcal{O}\stackrel{{\scriptstyle s}}{{\sim}}\mathcal{O}^{\prime}\sim\mathcal{O}_{par}$.

• (ii)

$\mathcal{O}\stackrel{{\scriptstyle s}}{{\sim}}\mathcal{O}_{i}$, $i\in\{113,\ldots,120\}$, and there is \mathcal{O}^{\prime\prime}\in\mbox{\Large\mathcal{O}}(\mathbf{P}_{4}) such that

[TABLE]

In both cases we use Corollary 7 to obtain relation (4.4).

Summing up, for $n=4$ and for every cyclic Jacobi method it holds

[TABLE]

For certain cyclic Jacobi methods better estimates hold. They are given by the relations (4.1) – (4.3).

Remark 8**.**

The results of this paper can be generalized to the block methods where an $n\times n$ matrix is partitioned in $\frac{n}{4}\times\frac{n}{4}$ blocks. For all cyclic strategies apart from those that are weakly equivalent to the parallel strategies from Subsection 3.3, one should only check that the assumptions from [7] are met (see [7], Theorem 3.11). For the strategies weakly equivalent to the parallel ones, Proposition 6 and Corollary 7 should be established for the block Jacobi method, with the constant $\gamma_{n}^{\prime}<1$, following the ideas from [2].

5. An application to $J$-Jacobi method

The results obtained in this paper have applications to different Jacobi-type methods. Here, we apply it to the $J$-Jacobi method from [15, 9].

The $J$-Jacobi method is an iterative method for solving the generalized eigenvalue problem $Ax=\lambda Jx$, $x\neq 0$, where $A$ is symmetric, $J$ is diagonal matrix of signs, and the pair $(A,J)$ is definite. Its main application is as a part of the highly accurate composite algorithm for the eigenvalue problem $Hx=\lambda x$, $x\neq 0$, with indefinite symmetric matrix $H$. Roughly speaking, after decomposing $H$ into $LJL^{T}$ by the Bunch-Parlett algorithm, the eigenvalue problem for $H$ is transformed into $Ax=\lambda Jx$, where $A=L^{T}L$ and $J$ is the diagonal matrix of signs which reveals the inertia of $H$ (see [9] for the details). The $J$-Jacobi method, which has been proposed by Veselić [15], applies $J$-orthogonal transformation matrices to $A$ in order to diagonalize it. Once $A$ is diagonalized by some $J$-orthogonal matrix $F$, that is $F^{T}AF=\Lambda_{A}$, $F^{T}JF=J$, the eigenvalues (eigenvectors) of $H$ are obtained as the diagonal elements of $J\Lambda_{A}$ (the columns of $L^{-1}JF\Lambda_{A}^{1/2}$). In practical computation, one-sided version of the $J$-Jacobi method is applied to $L$, but here we consider the two-sided method since the global convergence of the one-sided method reduces to the global convergence of the two-sided method.

Each iteration has the form

[TABLE]

where $A$ is symmetric positive definite. The transformation matrices are $J$-orthogonal, which means that $F_{k}^{T}JF_{k}=J$, $k\geq 0$. In particular, each $F_{k}$ is either a “trigonometric Jacobi rotation” or a “hyperbolic Jacobi rotation” in which hyperbolic sine and cosine appear. The corresponding step will be called trigonometric or hyperbolic.

The $J$-Jacobi method is said to be globally convergent if, for any initial $A$, the sequence of matrices $(A^{(k)},k\geq 0)$ generated by the iterative process (5.1) converges to some diagonal matrix. Again, it is sufficient to prove that $S(A^{(k)})\rightarrow 0$ as $k\rightarrow\infty$. Then the diagonal elements will converge (see the proof of [8, Theorem 3.7]). We note that the quadratic convergence for the serial method has been proved in [3].

We are concerned with $J$-symmetric $4\times 4$ matrices. Such matrices appear naturally as pivot submatrices in the block $J$-Jacobi method with $2\times 2$ blocks (or all column widths equal to $2$). Hence, we shall assume that $J=\text{diag}(1,1,-1,-1)$. (The cases $J=\text{diag}(1,-1,-1,-1)$ and $J=\text{diag}(1,1,1,-1)$ require somewhat different approach.)

Corollary 9**.**

Let $A$ be a symmetric $4\times 4$ matrix and $J=\text{diag}(1,1,-1,-1)$. Then every cyclic $J$-Jacobi method for the pair $(A,J)$ is globally convergent.

Proof.

Denote the pivot strategy by $I_{\mathcal{O}}$. We distinguish two cases: (i) $\mathcal{O}$ is generalized serial and (ii) $\mathcal{O}$ is weakly equivalent to $\mathcal{O}_{\text{par}}$ or $\mathcal{O}_{\text{par}^{\prime}}$.

For the case $(i)$ the proof follows directly from [7, Theorem 5.4], by taking $n=4$, $\nu=2$ and $\pi=(1,1,1,1)$. Proposition 5 (or [7, Theorem 3.11]) is used to ensure that every generalized serial ordering is convergent.

For the case $(ii)$ the proof is different. It uses two facts.

The first one is that the sequence $[\mathcal{O},\mathcal{O}]$ contains the sequence $\tilde{\mathcal{O}}$ which is equivalent to either $(1,2),(3,4),(1,3),(2,4),(1,4),(2,3)$ or $(1,2),(3,4),(1,4),$ $(2,3),(1,3),(2,4)$. We shall consider the first case for $\tilde{\mathcal{O}}$; the proof for the second case is quite similar.

The second fact reads: the “hyperbolic angle” $\theta_{k}$ of the hyperbolic rotation $F_{k}$ tends to [math] as $k$ increases [15, Lemma 2.2]. That result holds for general $n$ and for any pivot strategy. This implies that for every $\varepsilon>0$ there is $k_{0}\in\mathbb{N}$ such that, for $k\geq k_{0}$, the moduli of all pivot elements $|a_{i(k)j(k)}^{(k)}|$ associated with hyperbolic steps and all $|\tanh\theta_{k}|$ are smaller than $\varepsilon/2$. Since the pivot strategy $I_{\tilde{\mathcal{O}}}$ is parallel, we can write this condition in the form: for every $\varepsilon>0$ there is $r_{0}\in\mathbb{N}$ such that, for every $r\geq r_{0}$,

[TABLE]

We shall prove that the relations (5.2) and (5.3) imply $\lim_{r\rightarrow\infty}S(A^{(2r)})=0$. It is enough to prove that for every $\varepsilon$,

[TABLE]

there is $r_{0}$ such that $S(A^{(2r)})<\varepsilon$, for all $r\geq r_{0}$.

Let us simplify the notation. For given $r$, $r\geq r_{0}$, denote

[TABLE]

Then, we write

[TABLE]

It holds

[TABLE]

It is easy to to prove that

[TABLE]

Note that $a_{14}^{2}+a_{23}^{2}=0$ would imply $S(A^{(6r+4)})=0$. Using the relations (5.3) and (5.4) we obtain

[TABLE]

Hence,

[TABLE]

Moreover, note that $(a_{12}^{\prime})^{2}+(a_{34}^{\prime})^{2}=0$ would imply $S(A^{(6r+6)})=0$ and in the same way we have

[TABLE]

We use the relations (5.2), (5.4) and (5.5) – (5.8) to obtain

[TABLE]

Now, using (5.4), we can find bounds for $S(A^{(6r+2l)})$, $1\leq l\leq 4$,

[TABLE]

Since $r\geq r_{0}$ is arbitrary, the last set of relations implies that all $S(A^{(6r_{0}+2l)})$, $l\geq 0$, are smaller than $\varepsilon$. Thus, all elements of the sequence $S(A^{(2r)})$, $r\geq r_{0}$, except finitely many of them, are within $\varepsilon$-ball around [math], which proves the theorem.

Further more, the latest relations indicate the ultimate cubic convergence per cycle, irrespective of the multiplicities of the eigenvalues. ∎

We end the paper by addressing less interesting cases which we depict as

[TABLE]

We can first consider the process on \displaystyle\left[\begin{array}[]{rrr}\Box&\cdot&\cdot\\ \cdot&\Box&\cdot\\ \cdot&\cdot&\Box\end{array}\right]. It is a cyclic Jacobi method with perturbations coming from the hyperbolic transformations. Using the theory from [7, Section 5] one can prove that the off-norm of that $3$ by $3$ symmetric matric will tend to zero. Namely, when $n=3$ every pivot ordering belongs to the class $\mathcal{C}_{sp}^{(3)}$. The rest of the proof uses the fact that both, hyperbolic angles and the corresponding pivot elements tend to zero.

Acknowledgements

The authors are thankful to the anonymous reviewers for their suggestions and comments.