New deterministic approaches to the least square mean

TL;DR

This paper introduces new deterministic methods for approximating the least square mean of positive definite matrices, using inductive means of permuted matrix blocks, with proven convergence rates.

Contribution

It proposes a novel deterministic approximation technique for the geometric mean of positive definite matrices based on block permutations and inductive means, extending prior results.

Findings

Approximate least square mean using block permutation inductive means.

Proven convergence of the approximation to the true mean.

Provided rate of convergence estimate.

Abstract

In this paper we presents new deterministic approximations to the least square mean, also called geometric mean or barycenter of a finite collection of positive definite matrices. Let A1, A2, ..., Am be any elements of Md(C)+, where the set Md(C)+ is the open cone in the real vector space of selfadjoint matrices H(n). We consider a sequence of blocks of m matrices, that is,(A1, ..., Am, A1, ...,Am, A1, ...,Am, ...). We take a permutation on every block and then take the usual inductive mean of that new sequence. The main result of this work is that the inductive mean of this block permutation sequence approximate the least square mean on Md(C)+. This generalizes a Theorem obtain by Holbrook. Even more, we have an estimate for the rate of convergence.

Equations

$$(A_1,\dots ,A_m,A_1,\dots ,A_m,A_1,\dots A_m,\dots ).$$

$$L(\alpha )={\int }_a^b\Vert {\alpha }^{-1/2}(t){\alpha }^{\prime }(t){\alpha }^{-1/2}(t){\Vert }_{2}dt,$$

$$\delta(A,B)=\inf\{L(\alpha):\ \mbox{$\alpha$ is a piecewise smooth path connecting $A$ with $B$}\}.$$

$${\gamma }_{AB}(t)=A^{1/2}(A^{-1/2}B{A}^{-1/2}{)}^t{A}^{1/2}.$$

$${\delta }^2(X{\#}_{\frac{1}{2}}Y,Z)\le \frac{1}{2}{\delta }^2(X,Z)+\frac{1}{2}{\delta }^2(Y,Z)-\frac{1}{4}{\delta }^2(X,Y).$$

$${\delta }^2(X{\#}_{t}Y,Z)\le (1-t){\delta }^2(X,Z)+t{\delta }^2(Y,Z)-t(1-t){\delta }^2(X,Y).$$

$$f(t)=\delta (A{\#}_t{A}^{{}^{\prime }},B{\#}_t{B}^{{}^{\prime }})$$

$$\delta (A{\#}_t{A}^{{}^{\prime }},B{\#}_t{B}^{{}^{\prime }})\le (1-t)\delta (A,B)+t\delta (A^{{}^{\prime }},B^{{}^{\prime }}).$$

$$A\#_{\frac{1}{2}}B=\operatorname*{argmin}_{C\in\mathcal{M}_{d}(\mathbb{C})^{+}}\ \Big{(}\delta^{2}(A,C)+\delta^{2}(B,C)\,\Big{)}.$$

$$G(A_{1},\ldots,A_{m}):=\operatorname*{argmin}_{C\in\mathcal{M}_{d}(\mathbb{C})^{+}}\ \Big{(}\sum_{j=1}^{m}\delta^{2}(A_{j},C)\,\Big{)}.$$

$${\delta }^2(Z,G)\le \frac{1}{m}\left(\sum _{j=1}^m{\delta }^2(Z,A_j)-{\delta }^2(G,A_j)\right).$$

$$\mu =\frac{1}{m}\sum _{j=1}^m{\delta }_{\{A_j\}}.$$

$$S_1(A)$$

$$S_n(A)$$

$$A_{n}=A_{k}\quad\mbox{if $n\equiv k$ $\mod d$}.$$

$$\lim_{n\to\infty}S_{n}({A}_{\mbox{\tiny{${cyclic}$}}})=G(A_{0},\ldots,A_{d-1}).$$

$$A_{(r)}=\big{(}A_{r(1)},A_{r(2)},A_{r(3)},A_{r(4)},\ldots\big{)}$$

$$\underset{n\to \infty }{\lim }{S}_n(A_{(r)})=G(A_0,\dots ,A_{d-1}).$$

$$\underset{n\to \infty }{\lim }{S}_n(A)=G(A_0,\dots ,A_{d-1}).$$

$${\text{A}}_{\sigma }=(A_{{\sigma }_1(0)},\dots ,A_{{\sigma }_1(m-1)},A_{{\sigma }_2(0)},\dots ,A_{{\sigma }_2(m-1)},A_{{\sigma }_3(0)},\dots ,A_{{\sigma }_3(m-1)},\dots ),$$

$${\delta }^2(S_n({\text{A}}_{\sigma }),G)\le \frac{L}{n}.$$

$$\underset{n\to \infty }{\lim }{S}_n({\text{A}}_{\sigma })=G.$$

$${\text{A}}_{\sigma }^{(k)}=(\underset{kmmatrices}{\underbrace{A_{{\sigma }_1(1)},A_{{\sigma }_1(2)},\dots ,A_{{\sigma }_1(m)},\dots ,A_{{\sigma }_1(1)},A_{{\sigma }_1(2)},\dots ,A_{{\sigma }_1(m)}}},A_{{\sigma }_2(1)},\dots )$$

$${\delta }^2(S_n({\text{A}}_{\sigma }^{(k)}),G(A_1,\dots ,A_m))\le \frac{L}{n}.$$

$$G(\underset{ktimes}{\underbrace{A_1,\dots ,A_m,A_1,\dots ,A_m,\dots ,A_1,\dots ,A_m}})=G(A_1,\dots ,A_m).$$

$$\delta (S_n(A),S_n(B))\le \frac{1}{n}\sum _{i=1}^n\delta (A_i,B_i).$$

$${\delta }^2(S_{k+m}(A),Z)$$

$$-\frac{k}{(k+m{)}^2}\sum _{j=0}^{m-1}{\delta }^2(S_{k+j}(A),A_{k+j+1}).$$

$$(n+1){\delta }^2(S_{n+1}(A),Z)-n{\delta }^2(S_n(A),Z)$$

$$(k+m){\delta }^2(S_{k+m}(A),Z)-k{\delta }^2(S_k(A),Z),$$

$$\sum _{j=0}^{m-1}\left({\delta }^2(A_{k+j+1},Z)-\frac{k+j}{(k+j+1)}{\delta }^2(S_{k+j}(A),A_{k+j+1})\right).$$

Full text

[TABLE]

We take a permutation on every block and then take the usual inductive mean of that new sequence. The main result of this work is that the inductive mean of this block permutation sequence approximate the least square mean on $\mathcal{M}_{d}(\mathbb{C})^{+}$. This generalizes a Theorem obtain by Holbrook. Even more, we have an estimate for the rate of convergence.

1. 1 Introduction

2. 1.1 Setting of the problem

3. 1.2 The problem

4. 1.3 The main result of this paper

5. 2 Proof of main result

6. 3 Main theorem in Hadamard spaces

1 Introduction

1.1 Setting of the problem

Let $\mathcal{M}_{d}(\mathbb{C})^{+}$ denote the set of (strictly) positive matrices, which is an open cone in the real vector space of selfadjoint matrices $\mathcal{H}(n)$. In particular, it inherits a differential structure where the tangent spaces can be identified with $\mathcal{H}(n)$. The manifold $\mathcal{M}_{d}(\mathbb{C})^{+}$ can be endowed with a natural Riemannian structure such that the natural action of the invertible matrices by conjugations becomes isometric. With respect to this metric structure, if $\alpha:[a,b]\to\mathcal{M}_{d}(\mathbb{C})^{+}$ is a piecewise smooth path, its length is defined by

[TABLE]

where $\|\cdot\|_{2}$ denotes the Frobenius or Hilbert-Schmidt norm. As usual, a distance $\delta$ can be defined by

[TABLE]

The infimum is actually a minimum, and the geodesic connecting two positive matrices $A$ and $B$ has the following simple expression

[TABLE]

It is usual in matrix analysis to use the notation $A\#_{t}B$ instead of $\gamma_{AB}(t)$. The midpoint $A\#_{\frac{1}{2}}B$ is called geometric mean or barycenter between $A$ and $B$.

With the aforementioned Riemannian structure, $\mathcal{M}_{d}(\mathbb{C})^{+}$ becomes a Riemannian manifold with non-positive curvature. In particular, the distance function satisfies the so called semiparallelogram law.

Proposition 1.1**.**

Let $X,,Y$ be two matrices in $\mathcal{M}_{d}(\mathbb{C})^{+}$. Then for any $Z$ in $\mathcal{M}_{d}(\mathbb{C})^{+}$

[TABLE]

Inductively for all dyadic rationals $t\in[0,1]$, and then by continuity we get the following inequality for other points in the geodesic:

[TABLE]

As a consequence of this inequality we get that the function

[TABLE]

is convex $[0,1]$ for any $A,A^{\prime},B,B^{\prime}\in\mathcal{M}_{d}(\mathbb{C})^{+}$, more precisely

[TABLE]

From the semiparallelogram law, we obtain the following alternative characterization of the geometric mean

[TABLE]

There is no reason to restrict our attention to only two matrices. The notion of geometric mean can be generalized for more than two matrices in the obvious way

[TABLE]

The solution of this least square minimization problem exists and is unique because of the convexity properties of the distance $\delta(\cdot,\cdot)$. A multivariable version of the semiparallelogram law holds.

Proposition 1.2**.**

Let $A_{1},A_{2},\ldots,A_{m}$ be any elements of $\mathcal{M}_{d}(\mathbb{C})^{+}$ and let $G=G(A_{1},\ldots,A_{m})$. Then for all $Z\in\mathcal{M}_{d}(\mathbb{C})^{+}$ we have

[TABLE]

It is also known as the variance inequality, since $G(A_{1},\ldots,A_{m})$ can be interpreted as a nonlinear expectation of the probability measure

[TABLE]

The geometric means naturally appear in many applied problems. For instance, they appear in the study of radar signals (see [7] and the references therein for more details). Another typical application of the geometric means is in problems related with the gradient or Newton like optimization methods (see [6],[17]).

1.2 The problem

The usual problem dealing with geometric means is that the geometric mean of three or more matrices does not have in general a closed formula (see [5], [8], [9] and [14]). In [8], Holbrook proved that they can be approximated by the so called inductive means.

Definition**.**

Given a sequence of (strictly) positive matrices $A=(A_{n})_{n\in\mathbb{N}}$, the inductive means are defined as follows:

[TABLE]

Let $A_{0},\ldots,A_{d-1}$ be positive matrices, and define the cyclic sequence {A}_{\mbox{\tiny{{cyclic}}}}=(A_{n})_{n\in\mathbb{N}}, where for $n\geq d$ we define

[TABLE]

Then, the main result in [8] is the following

Theorem 1.3** (Holbrook).**

[TABLE]

1.3 The main result of this paper

The main advantage of Holbrook’s result is that it is deterministic. The results in [9] and [5] say that, if we consider the uniform distribution in $\{0,\ldots,d-1\}$ and we construct a sequence

[TABLE]

taking randomly the matrices $A_{0},\ldots,A_{d-1}$, then almost surely

[TABLE]

Although this implies that there are plenty of such a sequences, except for the cyclic one {A}_{\mbox{\tiny{{cyclic}}}}, we do not know if given a specific sequence $A$ we have that

[TABLE]

The main result of this paper allows to enlarge the set of deterministic examples of sequences $A$ such that (5) holds. So, fix positive matrices $A_{0},\ldots,A_{m-1}$, and consider a sequence of permutations of $m$-elements $\sigma=\{\sigma_{j}\}_{j\in\mathbb{N}}$. Then, define the sequence

[TABLE]

Using this notation, the main theorem of this paper is the following:

Theorem 1.4**.**

If $n\geq 1$ and $G=G(A_{0},\ldots,A_{m-1})$, then there exists $L>0$ depending on these matrices such that

[TABLE]

In particular

[TABLE]

Holbrook’s rate of convergence is $1/n$ as $n\rightarrow\infty$. Actually, as it was pointed out by Lim and Pálfia [16], one cannot expect better convergence rates than $1/n$. For that purpose considerer the case when $m=2$. So our rate of convergence should be able to be improved.

Finally, note that the above result is also true if we permutes blocks of length $km$ matrices for some $k\in\mathbb{N}$. More precisely,

Corollary 1.5**.**

Let $A_{1},\ldots,A_{m}\in\mathcal{M}_{d}(\mathbb{C})^{+}$ be positive matrices. Given $k\geq 1$, consider the sequence

[TABLE]

where each $\sigma$ is a permutation of $km$-elements. Then, there exists $L>0$ such that

[TABLE]

Indeed, note that by definition

[TABLE]

2 Proof of main result

In this section we are dedicated to prove Theorem 1.4. We begin with some basic fact about the inductive mean which is a direct consequence of (3).

Lemma 2.1**.**

Given two sequence $A=(A_{i})_{i\in\mathbb{N}},B=(B_{i})_{i\in\mathbb{N}}$ in $\mathcal{M}_{d}(\mathbb{C})^{+}$, then

[TABLE]

Now we will follow the work of Lim and Pálfia [15]. This first lemma is actually step 1 in their paper.

Lemma 2.2**.**

Given a sequence $A=(A_{i})_{i\in\mathbb{N}}$ in $\mathcal{M}_{d}(\mathbb{C})^{+}$ and $Z\in\mathcal{M}_{d}(\mathbb{C})^{+}$, for every $k,m\in\mathbb{N}$

[TABLE]

Proof.

By the inequality (2) applied to $S_{n+1}(A)=S_{n}(A)\,\#_{\frac{1}{n+1}}\,A_{n+1}$ we obtain

[TABLE]

Summing these inequalities from $n=k$ until $n=k+m-1$ we get that the difference

[TABLE]

obtained from the telescopic sum of the left hand side, is less or equal than

[TABLE]

Finally, using that $\frac{k+j}{k+j+1}\geq\frac{k}{k+m}$ for every $j\in\{0,\ldots,m-1\}$, this sum is bounded from the above by

[TABLE]

which completes the proof. ∎

Using the previous result we have this particular case:

Lemma 2.3**.**

Let $A_{1},\ldots,A_{m}\in\mathcal{M}_{d}(\mathbb{C})^{+}$ fixed. For every sequence $\textbf{A}_{\sigma}$,

[TABLE]

Proof.

Just change $k$ with $km$, $Z$ with $G$ and $A$ with $\textbf{A}_{\sigma}$ in Lemma 2.2.

∎

The next step is to find a lower bound for

[TABLE]

This is step 2 in [15] and here it is a little different.

Let $A_{1},\ldots,A_{m}\in\mathcal{M}_{d}(\mathbb{C})^{+}$ fixed, we will denote

[TABLE]

Note that by (2), for every sequence $\textbf{A}_{\sigma}$, all $k\in\mathbb{N}$ and all $j\in\mathbb{N}$,

[TABLE]

Lemma 2.4**.**

Let $A_{1},\ldots,A_{m}\in\mathcal{M}_{d}(\mathbb{C})^{+}$ fixed. For every sequence $\textbf{A}_{\sigma}$ and for all $k\in\mathbb{N}$ we have

[TABLE]

Proof.

Let $0\leq j\leq m-1$. Note that by (2) and all $k$,

[TABLE]

Hence

[TABLE]

Therefore, for every $j\leq m$,

[TABLE]

where we have used that $\delta(S_{km+j}(\textit{{A}}_{\sigma}),(\textbf{A}_{\sigma})_{km+j+1})\leq\Delta$ for every $k,j\in\mathbb{N}$. Summing up these inequalities and dividing by $m$, we get

[TABLE]

By the variance inequality (4),

[TABLE]

Note that

[TABLE]

So, combining (8) and (9) we get the desired result. ∎

Remark 2.5**.**

In equality (10) is essential that the matrices that appear in the first block $A_{\sigma_{1}(1)},\ldots,\\ A_{\sigma_{1}(m)}$ are the same as those that appear in the $(k+1)$-th block (except by the order). On the other hand, note that this result can not be extended to weighted means as those consider in [15]. Indeed, in that setting, the weights do not allow to consider permutations.

Now we prove Theorem 1.4 and the rate of convergence for a special subsequence.

Lemma 2.6**.**

Let $A_{1},\ldots,A_{m}\in\mathcal{M}_{d}(\mathbb{C})^{+}$ fixed. For every sequence $\textbf{A}_{\sigma}$ and for all $k\in\mathbb{N}$,

[TABLE]

where $L=\alpha+3\Delta^{2}$.

Proof.

We will prove it by induction. If $k=1$ the result is trivial because

[TABLE]

Let’s suppose that it’s true for $k$. Combining Lemmas 2.3 and 2.4 we get

[TABLE]

∎

To conclude the proof of Theorem 1.4. let $n=km+d$ such that $d\in\{1,\ldots,m-1\}$, $k\in\mathbb{N}$. Since $X\#_{t}X=X$ for all $X\in\mathcal{M}_{d}(\mathbb{C})^{+}$ and all $t\in[0,1]$, using Lemma 2.1 with the sequences

[TABLE]

and taking into account that $\delta(S_{km}(\textit{{A}}_{\sigma}),(\textbf{A}_{\sigma})_{km+j})\leq\Delta$ for every $j\in\{1,\ldots,d\}$, we get

[TABLE]

Combining this with Lemma 2.6 we obtain that for $n$ big enough $\delta(S_{n}(\textit{{A}}_{\sigma}),G)<\varepsilon$. $\square$

3 Main theorem in Hadamard spaces

In this section we will mention how all the previous result can be generalized to a much more general context. If we look in detail all the previous proofs, we can note that they are based mainly on the semiparallelogram law and the variance inequality (and their consequences). So, with the necessary chances, we can generalized the main theorem to non-positively curved (NPC) spaces, also called Hadamard spaces or (global) CAT(0) spaces. A complete metric space $(M,\delta)$ is called a Hadamard space if it satisfies the semiparallelogram law, i.e., for each $x,y\in M$ there exists $m\in M$ satisfying

[TABLE]

for all $z\in M$. The point $m$ is called (metric) midpoint between $x$ and $y$. Taking $z=x$ and $z=y$ in the inequality (11), it is easy to conclude that $\delta(x,m)=\delta(m,y)=\frac{1}{2}\delta(x,y)$. Moreover, this inequality also implies that the midpoint is unique. The existence and uniqueness of midpoints give rise to a unique (metric) geodesic $\gamma_{a,b}:[0,1]\rightarrow M$ connecting any given two points $a$ and $b$, that we denote as before as $a\#_{t}b$. The inequality (11) also extends to arbitrary points on geodesics.

[TABLE]

As in the case of strictly positive matrices, the inductive mean is defined in general Hadamard spaces in the same way (using the geodesic mention before). Also, the notion of barycenter and all the results that we use related to the barycenter (Existence and uniqueness - Variance Inequality) can be extend to this spaces. We refer the reader to [18], [19].

Acknowledgements:

The author wish to express their gratitude to Professor Jorge A. Antezana and Professor Demetrio Stojanoff fot theirs valuable comments and suggestions.