Generality of Lieb's Concavity Theorem

TL;DR

This paper extends Lieb's concavity theorem to a broader class of unitarily invariant, monotone, and concave matrix functions, demonstrating joint concavity under more general conditions.

Contribution

It generalizes Lieb's concavity theorem to any unitarily invariant, monotone, and concave matrix function, broadening its applicability.

Findings

Proves joint concavity of a generalized matrix function.

Extends Lieb's theorem to broader matrix functions.

Applicable for a wide range of parameters and functions.

Abstract

We show that Lieb's concavity theorem holds more generally for any unitarily invariant matrix function $\phi:\mathbf{H}^n_+\rightarrow \mathbb{R}$ that is monotone and concave. Concretely, we prove the joint concavity of the function $(A,B) \mapsto\phi\big[(B^\frac{qs}{2}K^*A^{ps}KB^\frac{qs}{2})^{\frac{1}{s}}\big] $ on $\mathbf{H}_+^m\times\mathbf{H}_+^n$, for any $K\in \mathbb{C}^{m\times n},s\in(0,1],p,q\in[0,1], p+q\leq 1$.

Equations

$$(A,B)⟼\mathrm{Tr}[K^{\ast }{A}^{p}K{B}^q]$$

$$(A,B)\ \longmapsto\ \phi\big{(}(B^{\frac{qs}{2}}K^{*}A^{ps}KB^{\frac{qs}{2}})^{\frac{1}{s}}\big{)},$$

$${\mathrm{Tr}}_k(A)=\sum _{1\le i_1<i_2<\cdots <i_k\le m}{\lambda }_{i_1}{\lambda }_{i_2}\cdots {\lambda }_{i_k},1\le k\le n,$$

$$X\mapsto \varphi (\mathcal{F}(X))$$

$${\mathbf{H}}^n$$

$$X\mapsto \sum _{i=1}^k{\lambda }_i^{\uparrow }(\mathcal{F}(X))$$

$$\sum_{i=1}^{k}\lambda^{\uparrow}_{i}\big{(}f(M^{*}AM)\big{)}=\inf_{\begin{subarray}{c}G\in\mathbb{C}^{n\times n},G^{2}=G\\ \mathrm{rank}(G)=k\end{subarray}}\mathrm{Tr}\big{[}f(M^{*}G^{*}AGM)\big{]},\quad\text{for any}\ A\in\mathbf{H}_{+}^{n},M\in\mathbb{C}^{n\times n}.$$

$$\mathbf{x}+\mathbf{y}=(x_1+y_1,\dots ,x_n+y_n),\mathbf{x}\mathbf{y}=(x_1{y}_1,\dots ,x_n{y}_n).$$

$$x_1^{\downarrow }\ge x_2^{\downarrow }\ge \cdots \ge x_n^{\downarrow },x_1^{\uparrow }\le x_2^{\uparrow }\le \cdots \le x_n^{\uparrow }.$$

$$f(\mathbf{x})=(f(x_1),\dots ,f(x_n)),x\in {\mathbb{R}}^n.$$

$$f(A)=\sum _{i=1}^{n}f({\lambda }_i(A))u_i{u}_i^{\ast },A\in {\mathbf{H}}^n,$$

$$\varphi (\mathbf{x})=\varphi (P\mathbf{x})$$

$$\mathbf{x}\ge \mathbf{y}$$

$$\varphi (\tau \mathbf{x}+(1-\tau )\mathbf{y})\le \tau \varphi (\mathbf{x})+(1-\tau )\varphi (\mathbf{y})$$

$$\varphi (A)=\varphi (\mathbf{\lambda }(A)),A\in {\mathbf{H}}^n.$$

$$\varphi (U^{\ast }AU)=\varphi (A)$$

$$A⪰B$$

$$\varphi (\tau A+(1-\tau )B)\le \tau \varphi (A)+(1-\tau )\varphi (B)$$

$$\varphi (\mathbf{x})=0⟺\mathbf{x}=(0,0,\dots ,0),$$

$${\mathrm{Tr}}_k[\mathbf{x}{]}^{\frac{1}{k}}={\left(\sum _{1\le i_1<i_2<\cdots <i_k\le n}{x}_{i_1}{x}_{i_2}\cdots x_{i_k}\right)}^{\frac{1}{k}},\mathbf{x}\in {\mathbb{R}}_{+}^n,1\le k\le n.$$

$$g_k(\mathbf{x})=\sum _{1\le i_1<i_2<\cdots <i_k\le n}(x_{i_1}{x}_{i_2}\cdots x_{i_k}{)}^{\frac{1}{k}},\mathbf{x}\in {\mathbb{R}}_{+}^n,1\le k\le n.$$

$$\Vert \mathbf{x}{\Vert }_p={\left(\sum _{i=1}^n{x}_i^p\right)}^{\frac{1}{p}},\mathbf{x}\in {\mathbb{R}}_{+}^n.$$

$$⟨{\mathbf{a}}^{\downarrow },{\mathbf{x}}^{\uparrow }⟩=\sum _{i=1}^k{a}_i^{\downarrow }{x}_i^{\uparrow },\mathbf{x}\in {\mathbb{R}}^n.$$

$$⟨\text{mathbbm}{1}_{\{i\le k\}},{\mathbf{x}}^{\uparrow }⟩=\sum _{i=1}^k{x}_i^{\uparrow },\mathbf{x}\in {\mathbb{R}}^n.$$

$$(A,B)\ \longmapsto\ \phi\big{(}(B^{\frac{qs}{2}}K^{*}A^{ps}KB^{\frac{qs}{2}})^{\frac{1}{s}}\big{)}$$

$$(A_{1},A_{2},\dots,A_{m})\ \longmapsto\ \phi\big{(}\exp\big{(}H+\sum_{j=1}^{m}p_{j}\log A_{j}\big{)}\big{)}$$

$$\sum_{i=1}^{k}\lambda^{\uparrow}_{i}\big{(}f(M^{*}AM)\big{)}=\inf_{\begin{subarray}{c}G\in\mathbb{C}^{n\times n},G^{2}=G\\ \mathrm{rank}(G)=k\end{subarray}}\mathrm{Tr}\big{[}f(M^{*}G^{*}AGM)\big{]}.$$

$$\sum_{i=1}^{k}\lambda^{\uparrow}_{i}\big{(}f(A)\big{)}=\inf_{\begin{subarray}{c}H\in\mathbf{H}^{n}\\ \mathrm{rank}(H)=n-k\end{subarray}}\mathrm{Tr}\big{[}f(H+A)\big{]}.$$

$$\sum _{i=m_1+1}^{m_2}{\lambda }_i^{\downarrow }(A)=$$

$$=$$

$$\sum _{i=1}^k{a}_i^{\downarrow }\le \sum _{i=1}^k{b}_i^{\downarrow },1\le k\le n;$$

Full text

1 Introduction

Lieb’s Concavity Theorem [2], as one of the most celebrated results in the study of trace inequalities, states that the function

[TABLE]

is jointly concave on $\mathbf{H}_{+}^{m}\times\mathbf{H}_{+}^{n}$, for any $K\in\mathbb{C}^{m\times n}$, $p,q\in(0,1],p+q\leq 1$. Here $\mathbf{H}_{+}^{n}$ is the convex cone of all $n\times n$ Hermitian, positive semidefinite matrices. Among rich consequences of the Lieb’s concavity theorem, the concavity of the map $A\mapsto\mathrm{Tr}[\exp(H+\log A)]$ on $\mathbf{H}_{+}^{n}$ and the three-matrix extension of the Golden-Thompson inequality, both also established by Lieb [2], are most acknowledged. As an important application, Lieb and Ruskai [3] used these results to prove the strong subadditivity of quantum entropy.

Since its original establishment, the concavity of (1) has been discussed from various perspectives and proved alternatively using, for example, the theory of Herglotz functions (Epstein [4]), quadratic interpolations (Uhlmann [5], Kosaki [6]) and matrix tensors (Ando [7], Carlen [8], Nikoufar et al. [9]). Recently, Huang [1] generalized Lieb’s result to the concavity of

[TABLE]

on $\mathbf{H}_{+}^{m}\times\mathbf{H}_{+}^{n}$ for any $K\in\mathbb{C}^{m\times n}$, $s\in(0,1],p,q\in(0,1],p+q\leq 1$, with $\phi(\cdot)=(\mathrm{Tr}_{k}[\cdot])^{\frac{1}{k}},1\leq k\leq n$. Here the $k$-trace $\mathrm{Tr}_{k}(A)$ of a matrix $A\in\mathbb{C}^{n\times n}$ is defined as

[TABLE]

with $\bm{\lambda}(A)=(\lambda_{1},\lambda_{2},\dots,\lambda_{n})$ being the eigenvalues of $A$, counting multiplicities. Huang’s proof was a direct use of an operator interpolation technique by Stein [10], and hence also provided a new proof of the original Lieb’s concavity theorem. An application of Huang’s on $k$-trace generalization is to derive concentration estimates on partial spectral sums of random matrices [11], which extended Tropp’s master bounds [12, 13] from the largest (or smallest) eigenvalue to the sum of the $k$ largest (or smallest) eigenvalues. Later, Huang [14] strengthened his result by showing that the map (2) is jointly concave for arbitrary $\phi:\mathbf{H}^{n}_{+}\rightarrow\mathbb{R}$ that is unitary invariant, monotone(monotone increasing with respect to Löwner order), concave and satisfies Hölder’s inequality, i.e. $\phi(|AB|)\leq\phi(|A|^{p})^{\frac{1}{p}}\phi(|B|^{q})^{\frac{1}{q}},\forall p,q\in[1,+\infty],\frac{1}{p}+\frac{1}{q}=1$.

However, though the immediate operator interpolation arguments in Huang’s previous proof require $\phi$ to satisfy Hölder’s inequality, the final result actually does not. In this paper, we will further improve Huang’s results by removing the Hölder’s condition. More precisely, we will prove the concavity of (2) for arbitrary $\phi:\mathbf{H}^{n}_{+}\rightarrow\mathbb{R}$ that is unitary invariant, monotone and concave. We remark that for any symmetric function $\phi:\mathbb{R}^{n}_{+}\rightarrow\mathbb{R}$ that is monotone increasing (with respect to the standard vector partial order) and concave, its extension to $\mathbf{H}^{n}_{+}$ defined as $\phi(A)=\phi(\bm{\lambda}(A)),A\in\mathbf{H}^{n}_{+}$ is unitarily invariant, monotone and concave on $\mathbf{H}^{n}_{+}$.

The proof of our further generalization will be based on an observation that, given any function $\mathcal{F}:\Omega\rightarrow\mathbf{H}^{n}$ from a convex set $\Omega$ to the space of all Hermitian matrices $\mathbf{H}^{n}$,

[TABLE]

if and only if

[TABLE]

where $\lambda^{\uparrow}_{i}(A)$ denotes the $i_{\text{th}}$ smallest eigenvalue of $A\in\mathbf{H}^{n}$. That is to say, we only need to prove the concavity of $\eqref{eqt:generalLCT}$ for $\phi(X)=\sum_{i=1}^{k}\lambda^{\uparrow}_{i}(\mathcal{F}(X)),1\leq k\leq n$. This strategy shares the spirit of Ky Fan’s dominance theorem (e.g. see Theorem 7.4.8.4 in [15]): given any $A,B\in\mathbb{C}^{n\times n}$, $\|A\|\leq\|B\|$ for arbitrary unitarily invariant norm $\|\cdot\|$ if and only if the singular values of $A$ is weakly majorized by the singular values of $B$. A similar idea was adopted in a recent work by Hiai et al. [16], in which they used majorization theories to show that, to prove a class of integral inequality for arbitrary unitarily invariant norm of Hermitian matrices requires only to prove it for every Ky Fan $k$-norm(sum of the $k$-largest eigenvalues). Inspired by their work, we will also use techniques of majorization to prove our preceding observation. Then we will prove the concavity of $\eqref{eqt:generalLCT}$ for $\phi(X)=\sum_{i=1}^{k}\lambda^{\uparrow}_{i}(\mathcal{F}(X)),1\leq k\leq n$ based on a new variational form of the sum the $k$ smallest eigenvalues: for any $f:\mathbb{R}_{+}\rightarrow\mathbb{R}$ that is monotone increasing and satisfies $f(0)=0$, we have

[TABLE]

outline

The rest of the paper is organized as follows. Section 2 is devoted to introductions of general notations, the notion of symmetric forms and our main results. We will briefly review in Section 3 the theories of majorization and use them to prove a useful equivalence theorem. The proofs of our main theorems are presented in Section 4.

2 Notations and Main Results

2.1 General conventions

For any positive integers $n,m$, we write $\mathbb{C}^{n}$ for the $n$-dimensional complex vector spaces equipped with the standard $l_{2}$ inner products, and $\mathbb{C}^{m\times n}$ for the space of all complex matrices of size $m\times n$. Let $\mathbb{R}^{n},\mathbb{R}_{+}^{n},\mathbb{R}_{++}^{n}$ be $(-\infty,+\infty)^{n},[0,+\infty)^{n},(0,+\infty)^{n}$, respectively. Let $\mathbf{H}^{n}$ be the space of all $n\times n$ Hermitian matrices, $\mathbf{H}_{+}^{n}$ be the convex cone of all $n\times n$ Hermitian, positive semi-definite matrices, and $\mathbf{H}_{++}^{n}$ be the convex cone of all $n\times n$ Hermitian, positive definite matrices. We write $I_{n}$ for the identity matrix of size $n\times n$. We use $S_{n}$ to denote the symmetric group of all permutations of order $n$.

For any $\bm{x}=(x_{1},\dots,x_{n}),\bm{y}=(y_{1},\dots,y_{n})\in\mathbb{R}^{n}$, we write $\bm{x}+\bm{y}$ and $\bm{x}\bm{y}$ for the entry-wise sum and entry-wise product respectively, i.e.

[TABLE]

We say $\bm{x}\leq\bm{y}$ if $x_{i}\leq y_{i},i=1,\dots,n$. We will denote by $\bm{x}^{\downarrow}$ and $\bm{x}^{\uparrow}$ the descending reordering and ascending reordering of $x$, respectively. That is, there exist some permutations $P_{1},P_{2}\in S_{n}$ such that $\bm{x}^{\downarrow}=P_{1}\bm{x},\bm{x}^{\uparrow}=P_{2}\bm{x}$, and

[TABLE]

For any function scalar function $f:\mathbb{R}\rightarrow\mathbb{R}$, the extension of $f$ to a function from $\mathbb{R}^{n}$ to $\mathbb{R}^{n}$ is given by

[TABLE]

For any $A\in\mathbf{H}^{n}$, we use $\lambda_{1}(A),\lambda_{2}(A),\dots,\lambda_{n}(A)$ to denote all the eigenvalues of $A$ and write $\bm{\lambda}(A)=(\lambda_{1}(A),\dots,\lambda_{n}(A))\in\mathbb{R}^{n}$. We will be frequently using $\bm{\lambda}^{\downarrow}(A)$ and $\bm{\lambda}^{\uparrow}(A)$ as the descending ordering and ascending ordering, respectively, of the eigenvalues of $A$, i.e. $\lambda^{\downarrow}_{i}(A)$ is the $i_{\text{th}}$ largest eigenvalue of $A$, and $\lambda^{\uparrow}_{i}(A)$ is the $i_{\text{th}}$ smallest eigenvalue of $A$. For any scalar function $f:\mathbb{R}\rightarrow\mathbb{R}$, the extension of $f$ to a function from $\mathbf{H}^{n}$ to $\mathbf{H}^{n}$ is given by

[TABLE]

where $u_{1},u_{2},\cdots,u_{n}\in\mathbb{C}^{n}$ are the corresponding normalized eigenvectors of $A$. Then obviously, the spectrum of $f(A)$ is $f(\bm{\lambda}(A))$; and if $f$ is monotone increasing on $\mathbb{R}$, then $\lambda^{\downarrow}_{i}(f(A))=f(\lambda^{\downarrow}_{i}(A))$. One can find more discussions and analysis on matrix functions in [8, 17].

2.2 Symmetric forms

We start with symmetric functions on $\mathbb{R}^{n}$ defined as follows.

Definition 2.1**.**

A function $\phi:\mathbb{R}^{n}\rightarrow\mathbb{R}$ is a symmetric form if it is invariant under permutation:

[TABLE]

A symmetric form $\phi$ is monotone (increasing) if

[TABLE]

A symmetric form $\phi$ is convex, if

[TABLE]

A symmetric form $\phi$ is concave if $-\phi$ is convex.

The domain of a symmetric form $\phi$ can be naturally extended from $\mathbb{R}^{n}$ to $\mathbf{H}^{n}$, by feeding $\phi$ the eigenvalues of a matrix in $\mathbf{H}^{n}$.

Definition 2.2**.**

The extension of a symmetric form $\phi$ to $\mathbf{H}^{n}$ is defined as

[TABLE]

Proposition 2.3**.**

Let $\phi$ be a symmetric form on $\mathbb{R}^{n}$, then its extension to $\mathbf{H}^{n}$ is unitarily invariant:

[TABLE]

If $\phi$ is monotone, then its extension to $\mathbf{H}^{n}$ is monotone with respect to Löwner order:

[TABLE]

If $\phi$ is convex, then its extension to $\mathbf{H}^{n}$ is convex:

[TABLE]

The unitary invariance and the monotonicity inheriting property follow straightforward from definition. The proof of the convexity inheriting property requires the use of majorization between eigenvalues. We hence divert the proof of Proposition 2.3 to Section 3. Due to the inheriting properties, in what follows we will not distinguish between a symmetric form and its extension to Hermitian matrices. We remark that, in many cases, the domain of a symmetric may be restricted to smaller regions that are permutatively invariant (e.g. $\mathbb{R}_{+}^{n},\mathbb{R}_{++}^{n}$) or unitarily invariant (e.g. $\mathbf{H}_{+}^{n},\mathbf{H}_{++}^{n}$), for effectiveness of monotonicity or convexity.

Generally, if a symmetric form $\phi$ is convex, homogeneous of order 1 and positive definite, i.e.

[TABLE]

then $\phi$ is called a symmetric gauge function. A famous bijection theory of von Neumann [18] says that any unitarily invariant matrix norm on $\mathbf{H}^{n}$ is the extension of some symmetric gauge function on $\mathbb{R}^{n}$. In this paper, however, our main results are most related to symmetric forms that are monotone and concave. Some examples of such class of symmetric forms are listed below.

The k-trace introduced in [11]:

[TABLE] 2. 2.

The sum of rotated partial geometric means:

[TABLE] 3. 3.

The semi $p$-norm for $p\in(-\infty,0)\cup(0,1]$:

[TABLE] 4. 4.

The weighted sum biased to smaller entries: given any $\bm{a}\in\mathbb{R}^{n}$,

[TABLE]

In particular, the sum of the $k$ smallest entries:

[TABLE]

Obviously, any positive combination of a collection of monotone, concave symmetric forms is still a monotone, concave symmetric forms. Also, we can generate many more monotone, concave symmetric forms by simply compositing with monotone, concave functions, as stated in the following proposition.

Proposition 2.4**.**

Let $\phi:\mathbb{R}^{n}\rightarrow\mathbb{R}$ be a symmetric form, and $f:\mathbb{R}\rightarrow\mathbb{R}$ be a function.

If $\phi$ is monotone, and $f$ is monotone increasing over $\mathrm{range}(\phi)$, then $f\circ\phi$ is a monotone symmetric form. If $\phi$ is convex, and $f$ is monotone increasing and convex over $\mathrm{conv}(\mathrm{range}(\phi))$, then $f\circ\phi$ is a convex symmetric form.

• 2.

If $f$ is monotone increasing, and $\phi$ is monotone over $\mathrm{range}(f)^{n}$, then $\phi\circ f$ is a monotone symmetric form. If $f$ is convex, and $\phi$ is monotone and convex over $\mathrm{conv}(\mathrm{range}(f)^{n})$, them $\phi\circ f$ is a convex symmetric form.

Note that the trace function $\mathrm{Tr}$ is a monotone, convex and also concave symmetric form on $\mathbf{H}^{n}$. Then combining Proposition 2.3 and Proposition 2.4, we can conclude that for any monotone increasing function $f$ on $\mathbb{R}$, $\mathrm{Tr}[f(\cdot)]$ is monotone on $\mathbf{H}^{n}$; and for any convex (or concave) function $f$ on $\mathbb{R}$, $\mathrm{Tr}[f(\cdot)]$ is convex (or concave) on $\mathbf{H}^{n}$. Therefore we have provided an alternative proof for Theorem 2.10 in [8].

2.3 Main Results

Our main purpose is to generalize Lieb’s concavity theorems from trace to symmetric forms that are monotone and concave. Huang [1] applied operator interpolations to obtain generalizations of Lieb’s concavity to k-traces $\phi(x)=\mathrm{Tr}_{k}[x]^{\frac{1}{k}}$, which he used to derive concentration estimates on partial spectral sums of random matrices [11]. Since the interpolation part of Huang’s proof requires essentially the symmetry and Hölder property of the $k$-trace, his results can be extended to more general symmetric forms that are monotone, concave and satisfies Hölder’s inequality [14]. Even further, we find the Hölder property actually unnecessary, and by adopting techniques of majorization we can strengthen Huang’s results to the following.

Theorem 2.5** (General Lieb’s Concavity Theorem).**

Let $\phi$ be a symmetric form that is monotone and concave on $\mathbb{R}_{+}^{n}$. Then for any $K\in\mathbb{C}^{m\times n}$ and any $s\in(0,1],p,q\in[0,1],p+q\leq 1$, the function

[TABLE]

is jointly concave on $\mathbf{H}_{+}^{m}\times\mathbf{H}_{+}^{n}$.

Theorem 2.6**.**

Let $\phi$ be a symmetric form that is monotone and concave on $\mathbb{R}_{+}^{n}$. Then for any $H\in\mathbf{H}^{n}$ and any $\{p_{j}\}_{j=1}^{m}\subset[0,1]$ such that $\sum_{j=1}^{m}p_{j}\leq 1$, the function

[TABLE]

is jointly concave on $(\mathbf{H}_{++}^{n})^{\times m}$. In particular, A\mapsto\phi\big{(}\exp(H+\log A)\big{)} is concave on $\mathbf{H}_{++}^{n}$.

2.5 is a further generalization of the generalized Lieb’s concavity theorem (Theorem 3.2) in [1] (see also Theorem 2.5 in [14]), and 2.6 is a further generalization of Theorem 3.3 in [1] (see also Corollary 6.1 in [2] or Theorem 2.6 in [14]). We will first show that it is sufficient to prove the concavity of $\eqref{eqt:function1}$ and (4) with $\phi$ being the sum of the $k$ smallest eigenvalues for all $1\leq k\leq n$. This proof strategy is inspired by a recent work of Hiai et al. [16], in which they used majorization theories to generalize some multivariate trace inequalities. They showed that, to prove a class of integral inequality for arbitrary unitarily invariant norm of Hermitian matrices requires only to prove it for every Ky Fan $k$-norm, namely the sum of the $k$ largest singular values. Following their idea, we will also use techniques of majorization to first obtain an equivalence theorem as follows.

Theorem 2.7**.**

Let $\Omega$ be a convex set in some linear space, and $\mathcal{F}:\Omega\rightarrow\mathbf{H}^{n}$ be a function that maps $\Omega$ to $n\times n$ Hermitian matrices. Then the following two statements are equivalent:

• (i)

For any monotone, convex symmetric form $\phi$ on $\mathbb{R}^{n}$, the map X\mapsto\phi\big{(}\mathcal{F}(X)\big{)} is convex on $\Omega$.

• (ii)

For any $1\leq k\leq n$, the map X\mapsto\sum_{i=1}^{k}\lambda^{\downarrow}_{i}\big{(}\mathcal{F}(X)\big{)} is convex on $\Omega$.

Similarly, the following two statements are equivalent:

• (i*)

For any monotone, concave symmetric form $\phi$ on $\mathbb{R}^{n}$, the map X\mapsto\phi\big{(}\mathcal{F}(X)\big{)} is concave on $\Omega$.

• (ii*)

For any $1\leq k\leq n$, the map X\mapsto\sum_{i=1}^{k}\lambda^{\uparrow}_{i}\big{(}\mathcal{F}(X)\big{)} is concave on $\Omega$.

We remark that, the convex part and the concave part of 2.7 are equivalent. In fact, if the convex part is true, we can immediately prove the concave part by considering $\mathcal{F}(\cdot)\rightarrow-\mathcal{F}(\cdot)$, $\phi(\cdot)\rightarrow-\phi(-(\cdot))$ and noticing that $-\lambda^{\downarrow}_{i}(\mathcal{F}(\cdot))=\lambda^{\uparrow}_{i}(-\mathcal{F}(\cdot))$. The proof is diverted to the end of Section 3, after our brief review on some fundamental theories of majorization.

Supported by 2.7, we can confidently reduce our task to proving the concavity of (3) and (4) only for $\phi(\bm{x})=\sum_{i=1}^{k}x^{\uparrow}_{i},1\leq k\leq n$. This will be done by interpreting the sum of the k smallest eigenvalues as the infimum of some specialized trace functions, using the following two lemmas.

Lemma 2.8**.**

Let $f:\mathbb{R}_{+}\rightarrow\mathbb{R}$ be a monotone increasing function such that $f(0)=0$. Then for any $A\in\mathbf{H}_{+}^{n},M\in\mathbb{C}^{n\times n}$ and any $k\leq n$,

[TABLE]

Moreover, if $M$ is invertible, the infimum can be achieved.

Lemma 2.9**.**

Let $f:\mathbb{R}\rightarrow\mathbb{R}$ be a monotone increasing function such that $f(x\rightarrow-\infty)=0$. Then for any $A\in\mathbf{H}^{n}$ and any $k\leq n$,

[TABLE]

The proofs of Lemma 2.8 and Lemma 2.9 will be presented in Section 4, followed by the proofs of 2.5 and 2.6.

We will be using frequently the following extended version of the Courant-Fisher characterization (min-max theorem) for eigenvalues of Hermitian matrices. One may refer to [15, 19] for a proof.

Theorem 2.10** (Courant-Fisher).**

For any $A\in\mathbf{H}^{n}$ and any $0\leq m_{1}<m_{2}\leq n$,

[TABLE]

3 Majorization

For any two vectors $\bm{a},\bm{b}\in\mathbb{R}^{n}$, $\bm{a}$ is said to be weakly majorized by $\bm{b}$, denoted by $\bm{a}\prec_{w}\bm{b}$, if

[TABLE]

moreover, a is said to be majorized by $b$, denoted by $\bm{a}\prec\bm{b}$, if equality holds for $k=n$, i.e.

[TABLE]

The following two lemmas are most important for deriving inequalities from majorization relations. One may refer to [20, 21, 22] for proofs and more discussions on this topic.

Lemma 3.11**.**

For any $\bm{a},\bm{b}\in\mathbb{R}^{n}$, if $\bm{a}\prec_{w}\bm{b}$, then there is some $\bm{c}\in\mathbb{R}^{n}$ such that $\bm{a}\leq\bm{c}\prec\bm{b}$.

Lemma 3.12**.**

For any $\bm{a},\bm{b}\in\mathbb{R}^{n}$, $\bm{a}\prec\bm{b}$ if and only if $\bm{a}=D\bm{b}$ for some doubly stochastic matrix $D$, i.e. $D_{ij}\geq 0,1\leq i,j\leq n$, $\sum_{j=1}^{n}D_{ij}=1,1\leq i\leq n$ and $\sum_{i=1}^{n}D_{ij}=1,1\leq j\leq n$.

Since any doubly stochastic matrix is a convex combination of permutation matrices, an equivalent statement of Lemma 3.12 is that, $\bm{a}\prec\bm{b}$ if and only if $\bm{a}$ is a convex combination of permutations of $\bm{b}$, i.e.

[TABLE]

for some $\{P_{j}\}_{j=1}^{m}\subset S_{n}$ and some $\{\tau_{j}\}_{j=1}^{m}\subset[0,1]$ such that $\sum_{j=1}^{m}\tau_{j}=1$.

Lemma 3.13**.**

Let $\phi:\mathbb{R}^{n}\rightarrow\mathbb{R}$ be a convex symmetric form. Then for any $\bm{a},\bm{b}\in\mathbb{R}^{n}$, $\bm{a}\prec\bm{b}$ implies $\phi(\bm{a})\leq\phi(\bm{b})$. Moreover, if $\phi$ is also monotone, then $\bm{a}\prec_{w}\bm{b}$ implies $\phi(\bm{a})\leq\phi(\bm{b})$.

Proof.

If $\bm{a}\prec\bm{b}$, we have $\bm{a}=\sum_{j=1}^{m}\tau_{j}P_{j}\bm{b}$ for some permutations $\{P_{j}\}_{j=1}^{m}\subset S_{n}$ and some $\{\tau_{j}\}_{j=1}^{m}\subset[0,1]$ such that $\sum_{j=1}^{m}\tau_{j}=1$. Then using convexity and permutation invariance of $\phi$, we have

[TABLE]

If $\bm{a}\prec_{w}\bm{b}$, then by Lemma 3.11 there is some $\bm{c}\in\mathbb{R}^{n}$ such that $\bm{a}\leq\bm{c}\prec\bm{b}$. Then we have $\phi(\bm{c})\leq\phi(\bm{b})$. Moreover, if $\phi$ is monotone, we have $\phi(\bm{a})\leq\phi(\bm{c})$, and so $\phi(\bm{a})\leq\phi(\bm{b})$. ∎

The following lemma is a widely used fact on majorization relations between eigenvalues. We provide the proof here to illustrate the proof techniques for majorization relations.

Lemma 3.14**.**

For any $A,B\in\mathbf{H}^{n}$,

[TABLE]

Proof.

For any Hermitian matrix $A\in\mathbf{H}^{n}$ and any $1\leq k\leq n$, by 2.10 we have that

[TABLE]

Therefore, for any $A,B\in\mathbf{H}^{n}$, we have

[TABLE]

And obviously we have

[TABLE]

Therefore $\bm{\lambda}(A+B)\prec\bm{\lambda}(A)+\bm{\lambda}(B)$. ∎

Proof of Proposition 2.3.

$\phi(A)$ is only a symmetric function of the eigenvalues of $A$, and the eigenvalues of $A$ and $U^{*}AU$ are the same for any unitary $U$, we thus have $\phi(U^{*}AU)=\phi(A)$.

For any $A,B\in\mathbf{H}^{n}$, if $A\succeq B$, then $\bm{\lambda}^{\downarrow}(A)\geq\bm{\lambda}^{\downarrow}(B)$ by 2.10. Therefore if $\phi$ is monotone as a vector symmetric form, we have $\phi(A)=\phi(\bm{\lambda}^{\downarrow}(A))\geq\phi(\bm{\lambda}^{\downarrow}(B))=\phi(B)$.

For any $A,B\in\mathbf{H}^{n}$ and any $\tau\in[0,1]$, let $C=\tau A+(1-\tau)B$. By Lemma 3.12, we know that

[TABLE]

Then by Lemma 3.13 and the convexity of $\phi$ as a vector symmetric form, we immediately have that

[TABLE]

Therefore $\phi$ is also convex on $\mathbf{H}^{n}$. ∎

Proof of 2.7.

As we have mentioned, the convex part and the concave part of the theorem are equivalent. We thus only prove the equivalence between $(i\text{*})$ and $(ii\text{*})$. $(i\text{*})\Rightarrow(ii\text{*})$ is trivial, since $\phi_{k}(\bm{x})=\sum_{i=1}^{k}x^{\uparrow}_{i}$ is a monotone, concave symmetric form on $\mathbb{R}^{n}$ for each $1\leq k\leq n$.

Now suppose $(ii\text{*})$ is true. Then for any $X,Y\in\Omega$ and any $\tau\in[0,1]$, with $Z=\tau X+(1-\tau)Y$, we have

[TABLE]

Let \bm{a}=\tau\bm{\lambda}\big{(}\mathcal{F}(X)\big{)}+(1-\tau)\bm{\lambda}\big{(}\mathcal{F}(Y)\big{)}\in\mathbb{R}^{n}. Note that for any $1\leq i\leq n$,

[TABLE]

We thus have -\bm{\lambda}\big{(}\mathcal{F}(Z)\big{)}\prec_{w}-\bm{a}. Then by Lemma 3.11 and Lemma 3.12, there exist some $\bm{b}\in\mathbb{R}^{n}$ and some doubly stochastic matrix $D$ such that -\bm{\lambda}\big{(}\mathcal{F}(Z)\big{)}\leq-\bm{b}=D(-\bm{a}), or equivalently,

[TABLE]

Now for any monotone, concave symmetric form $\phi$ on $\mathbb{R}^{n}$, we have \phi\big{[}\bm{\lambda}\big{(}\mathcal{F}(Z)\big{)}\big{]}\geq\phi(\bm{b}) due to monotonicity; and $\phi(\bm{b})\geq\phi(\bm{a})$ due to concavity and Lemma 3.13. Also due to concavity of $\phi$ we have

[TABLE]

Therefore, by the definition we have

[TABLE]

which means X\mapsto\phi\big{(}\mathcal{F}(X)\big{)} is concave on $\Omega$. ∎

4 Proof of main theorems

We still need two more lemmas for the proof of our main results. In Lemma 2.8, the infimum is taken over all idempotent matrices of rank $k$. We thus need to use properties of this class of matrices. It is well known that if a matrix is idempotent, then its eigenvalues must be either 0 or 1. Moreover, the following lemma tells that the singular values of a idempotent matrix are either 0, or greater than or equal to 1.

Lemma 4.15**.**

Let $P\in\mathbb{C}^{n\times n}$ be idempotent, i.e. $P^{2}=P$. Then all non-zero singular values of $P$ are greater than or equal to 1.

Proof.

Let $P=U\Sigma V^{*}$ be the compact singular value decomposition of $P$, where $U,V\in\mathbb{C}^{n\times k}$ satisfy $U^{*}U=V^{*}V=I_{k}$, $\Sigma\in\mathbf{H}_{++}^{k}$ is diagonal, and $k=rank(P)$. We need to show that $\Sigma\succeq I_{k}$. Since $P$ is idempotent, we have

[TABLE]

Using $U^{*}U=V^{*}V=I_{k}$, we obtain that

[TABLE]

Then for any $x\in\mathbb{C}^{k}$, we have

[TABLE]

Therefore $\Sigma^{-1}\preceq I_{k}$, or equivalently, $\Sigma\succeq I_{k}$. ∎

The next lemma is a variation of the Courant-Fisher characterization.

Lemma 4.16**.**

Let $f:\mathbb{R}\rightarrow\mathbb{R}$ be monotone increasing. Then for any $A\in\mathbf{H}^{n}$ and any $1\leq k\leq n$,

[TABLE]

Proof.

We only prove the first identity in Equation 9. The proof of the second identity is totally parallel. For any $Q\in\mathbb{C}^{n\times k},Q^{*}Q=I_{k}$ and any $V\in\mathbb{C}^{k\times i},V^{*}V=I_{i}$ with $i\leq k\leq n$, we have $QV\in\mathbb{C}^{n\times i},(QV)^{*}QV=I_{i}$. Thus by 2.10 we have

[TABLE]

Since $f$ is monotone increasing, we obtain that

[TABLE]

In particular, if we choose $Q=[\bm{q}_{1},\dots,\bm{q}_{k}]\in\mathbb{C}^{n\times k}$ to be the orthonormal eigenvectors of $A$ corresponding to the eigenvalues $\lambda^{\downarrow}_{1}(A),\dots,\lambda^{\downarrow}_{k}(A)$, we have exactly \mathrm{Tr}[f(Q^{*}AQ)]=\sum_{i=1}^{k}\lambda^{\downarrow}_{i}\big{(}f(A)\big{)}. Therefore we have

[TABLE]

∎

Proof of Lemma 2.8.

Let $\mathcal{G}_{k}=\{G\in\mathbb{C}^{n\times n}:G^{2}=G,\mathrm{rank}(G)=k\}$. We first prove identity (5) with “$\inf$” replaced by “$\min$”, for any invertible $M$. We need to show that the inequality

[TABLE]

holds for any $G\in\mathcal{G}_{k}$. We define

[TABLE]

Since $G^{2}=G$, we have $P^{2}=M^{-1}GMM^{-1}GM=M^{-1}G^{2}M=P$. That is, P is idempotent. Also we have $\mathrm{rank}(P)=\mathrm{rank}(G)=k$. Let $P=U\Sigma V^{*}$ be the compact singular value decomposition of $P$, where $U,V\in\mathbb{C}^{n\times k}$ satisfy $U^{*}U=V^{*}V=I_{k}$, and $\Sigma\in\mathbf{H}_{++}^{k}$ is diagonal. By Lemma 4.15, we know $\Sigma\succeq I_{k}$. Then we have

[TABLE]

We have used the fact that $\mathrm{Tr}[f(X^{*}X)]=\mathrm{Tr}[f(XX^{*})]$ for any $X\in\mathbb{C}^{n\times m}$, since the spectrum of $X^{*}X$ and the spectrum of $XX^{*}$ may only differ by some zeros, but we have $f(0)=0$. Since $\Sigma\succeq I_{k}$ and $\Sigma$ is diagonal, we have $\Sigma^{2}\succeq I_{k}$, and thus

[TABLE]

Since $f$ is monotone increasing, by Proposition 2.4, $\mathrm{Tr}[f(\cdot)]$ is a monotone symmetric form. Therefore we obtain

[TABLE]

Again since $f$ is monotone increasing, by Lemma 4.16, we have

[TABLE]

So we have proved inequality (10). We then need to find some $G\in\mathcal{G}_{k}$ so that the equality in (10) holds. In fact, we can choose $G=MQQ^{*}M^{-1}$, where $Q=[\bm{q}_{1},\dots,\bm{q}_{k}]\in\mathbb{C}^{n\times k},Q^{*}Q=I_{k}$ and $\bm{q}_{i}$ is the normalized eigenvector of $M^{*}AM$ corresponding to the eigenvalue $\lambda^{\uparrow}_{i}(M^{*}AM)$. It is easy to see that $\mathrm{rank}(G)=k$ and $G^{2}=MQQ^{*}M^{-1}MQQ^{*}M^{-1}=MQQ^{*}M^{-1}=G$. Moreover, we have

[TABLE]

and thus

[TABLE]

Next, we will prove identity (5) for a general $M$ that is not necessarily invertible. For any $M\in\mathbb{C}^{n\times n}$, we can always find a sequence $\{M_{j}\}_{j=1}^{+\infty}\subset\mathbb{C}^{n\times n}$ such that (i) $M_{j}\rightarrow M$ entry-wisely as $j\rightarrow+\infty$, (ii) each $M_{j}$ is invertible, and (iii) $M_{j}M_{j}^{*}\succeq MM^{*}$. Such sequence $\{M_{j}\}_{j=1}^{+\infty}$ can be easily obtained by only modifying the singular values of $M$. Note that $f$ is continuous since it is convex; ordered eigenvalues and trace are also continuous on $\mathbf{H}^{n}$. Therefore, for any $G\in\mathcal{G}_{k}$, we have

[TABLE]

Moreover, for each $M_{j}$, there is some $G_{j}\in\mathcal{G}_{k}$ such that \mathrm{Tr}\big{[}f(M_{j}^{*}G_{j}^{*}AG_{j}M_{j})\big{]}=\sum_{i=1}^{k}\lambda^{\uparrow}_{i}\big{(}f(M_{j}^{*}AM_{j})\big{)}. Thus we have

[TABLE]

Then again since \sum_{i=1}^{k}\lambda^{\uparrow}_{i}\big{(}f(M^{*}AM)\big{)}=\lim_{j\rightarrow+\infty}\sum_{i=1}^{k}\lambda^{\uparrow}_{i}\big{(}f(M_{j}^{*}AM_{j})\big{)}, we must have

[TABLE]

and so identity (5) is proved. ∎

Proof of Lemma 2.9.

Since $f$ is monotone increasing on $\mathbb{R}$, we have $f(x)\geq f(-\infty)=0$ for all $x\in\mathbb{R}$, and thus $f(X)\in\mathbf{H}_{+}^{n}$ for all $X\in\mathbf{H}^{n}$. Let $\mathcal{H}_{k}=\{H\in\mathbf{H}^{n}:\mathrm{rank}(H)=n-k\}$. For any $H\in\mathcal{H}_{k}$, since $\dim\mathrm{Null}(H)=k$, we can always find some $U\in\mathbb{C}^{n\times k}$ such that $U^{*}U=I_{k}$ and $HU=\bm{0}$. Then by Lemma 4.16 we have

[TABLE]

Next we need to show that for arbitrary small $\epsilon>0$, there is some $H_{\delta}\in\mathcal{H}_{k}$ such that

[TABLE]

Let $A=Q\Lambda Q^{*}$ be the an eigenvalue decomposition of $A$, where $Q\in\mathbb{R}^{n\times n}$ is unitary, and $\Lambda$ is diagonal with ascending diagonal entries $\lambda^{\uparrow}_{1}(A),\dots,\lambda^{\uparrow}_{n}(A)$. We then take $H_{\delta}=Q\Lambda_{\delta}Q^{*}$, where $\Lambda_{\delta}$ is also diagonal, and the $i_{\text{th}}$ diagonal entry of $\Lambda_{\delta}$ is

[TABLE]

When $\delta$ is large enough, we can have $-\delta-\lambda^{\uparrow}_{i}(A)<0$ for all $k<i\leq n$, and thus $H_{\delta}\in\mathcal{H}_{k}$. And we have

[TABLE]

Since $f(x\rightarrow-\infty)=0$, we can always choose $\delta$ large enough so that $H_{\delta}\in\mathcal{H}_{k}$ and $(n-k)f(-\delta)\leq\epsilon$. So we have proved identity (6). ∎

Proof of 2.5.

We only need to show that

[TABLE]

is jointly concave on $\mathbf{H}_{+}^{m}\times\mathbf{H}_{+}^{n}$ for all $1\leq k\leq n$. According to Theorem 3.2 in [1], for any $L\in\mathbb{C}^{m\times n}$, the function

[TABLE]

is jointly concave on $\mathbf{H}_{+}^{m}\times\mathbf{H}_{+}^{n}$. Thus for any $A_{1},B_{1}\in\mathbf{H}_{+}^{m},A_{2},B_{2}\in\mathbf{H}_{+}^{n}$ and any $\tau\in[0,1]$, with $C_{i}=\tau A_{i}+(1-\tau)B_{i},i=1,2$, we have

[TABLE]

We have used formula (5) from Lemma 2.8 with $f(x)=x^{\frac{1}{s}}$, which is monotone increasing on $\mathbb{R}_{+}$ and satisfies $f(0)=0$. So we have proved the concavity of (11) for all $1\leq k\leq n$. The concavity of (3) then follows from 2.7 with

[TABLE]

∎

Proof of 2.5.

We only need to show that

[TABLE]

is jointly concave on $(\mathbf{H}_{++}^{n})^{\times m}$ for all $1\leq k\leq n$. According to Corollary 6.1 in [2](see also Theorem 3.3 in [1]), for any $L\in\mathbf{H}^{n}$, the function

[TABLE]

is jointly concave on $(\mathbf{H}_{++}^{n})^{\times m}$. Thus for any $(A_{1},A_{2},\dots,A_{m}),(B_{1},B_{2},\dots,B_{m})\in(\mathbf{H}_{++}^{n})^{\times m}$ and any $\tau\in[0,1]$, with $C_{i}=\tau A_{i}+(1-\tau)B_{i},i=1,\dots,m$, we have

[TABLE]

We have used formula (6) from Lemma 2.9 with $f(x)=\exp(x)$, which is monotone increasing on $\mathbb{R}$ and satisfies $f(x\rightarrow-\infty)=0$. So we have proved the concavity of (12) for all $1\leq k\leq n$. The concavity of (4) then follows from 2.7 with

[TABLE]

∎

Acknowledgment

The research was in part supported by the NSF Grant DMS-1613861. The author would like to thank Thomas Y. Hou for his wholehearted mentoring and supporting.