Bounds on Lyapunov exponents via entropy accumulation

TL;DR

This paper introduces entropy accumulation-based bounds for Lyapunov exponents of large random matrix products, providing analytical, tight bounds that are computationally efficient to evaluate.

Contribution

It establishes a novel link between Lyapunov exponents and entropy accumulation, enabling analytical bounds that are tight and easier to compute.

Findings

Bounds are tight in the commutative case.

Bounds are expressed as optimization problems involving single matrices.

Upper bounds can be efficiently computed via convex optimization.

Abstract

Lyapunov exponents describe the asymptotic behavior of the singular values of large products of random matrices. A direct computation of these exponents is however often infeasible. By establishing a link between Lyapunov exponents and an information theoretic tool called entropy accumulation theorem we derive an upper and a lower bound for the maximal and minimal Lyapunov exponent, respectively. The bounds assume independence of the random matrices, are analytical, and are tight in the commutative case as well as in other scenarios. They can be expressed in terms of an optimization problem that only involves single matrices rather than large products. The upper bound for the maximal Lyapunov exponent can be evaluated efficiently via the theory of convex optimization.

Equations

$$\mathbb{E}{\log }^{+}{\sigma }_{\max }(L_i)<\infty \text{for all }i\in [n]:=\{1,2,\dots ,n\},$$

$$\displaystyle\gamma_{k,n}:=\frac{1}{n}\mathbb{E}\log\sigma_{k}\Big{(}\prod_{i=1}^{n}L_{i}\Big{)}\quad\text{for}\quad 1\leq k\leq d\quad\text{and}\quad n\in\mathbb{N}\,,$$

$${\gamma }_k:=\underset{n\to \infty }{\lim }{\gamma }_{k,n}\text{for}1\le k\le d.$$

$$\displaystyle\gamma_{k}=\lim_{n\to\infty}\frac{1}{n}\log\sigma_{k}\Big{(}\prod_{i=1}^{n}L_{i}\Big{)}\quad\mathbb{P}-\text{a.s.}\quad\text{for}\quad 1\leq k\leq d\,.$$

$${\gamma }_1=\underset{n\to \infty }{\lim }\frac{1}{n}\mathbb{E}\log {\Vert \prod _{i=1}^n{L}_i\Vert }_p\text{for}p\ge 1,$$

$$2{\gamma }_1\le \underset{X\in {\mathbb{X}}_{\mathcal{S}}}{\max }\mathbb{E}\log \mathrm{tr}X{L}_1{L}_1^{†}\text{and}2{\gamma }_d\ge \underset{X\in {\mathbb{X}}_{\mathcal{G}}}{\min }\mathbb{E}\log \mathrm{tr}X{L}_1{L}_1^{†},$$

$$2{\gamma }_1\le \underset{X\in {\mathbb{X}}_{{\mathbb{C}}^{d\times d}}^{\prime }}{\max }\mathbb{E}\log \mathrm{tr}X{L}_1{L}_1^{†}\text{and}2{\gamma }_d\ge \underset{X\in {\mathbb{X}}_{{\mathbb{C}}^{d\times d}}^{\prime }}{\min }\mathbb{E}\log \mathrm{tr}X{L}_1{L}_1^{†},$$

$${\mathbb{X}}_{\mathcal{S}}:=\{Y^{†}Y/\mathrm{tr}{Y}^{†}Y:Y\in \mathcal{S}\}.$$

$${\mathbb{X}}_{{\mathbb{C}}^{d\times d}}^{\prime }:=\{Y^{†}Y/\mathrm{tr}{Y}^{†}Y:Y\in {\mathbb{C}}^{d\times d},\mathrm{rank}Y=1\}.$$

$${\lambda }_{\max }(L{L}^{†})=\underset{X\in {\mathbb{X}}_{{\mathbb{C}}^{d\times d}}}{\max }\mathrm{tr}XL{L}^{†}\text{and}{\lambda }_{\min }(L{L}^{†})=\underset{X\in {\mathbb{X}}_{{\mathbb{C}}^{d\times d}}}{\min }\mathrm{tr}XL{L}^{†}.$$

$${\lambda }_{\min }(L{L}^{†})\ge \frac{1}{d}\underset{Y\in \mathcal{G}}{\min }\frac{\mathrm{tr}{Y}^{†}YL{L}^{†}}{\mathrm{tr}{Y}^{†}Y}.$$

$${\sigma }_{\max }(L{)}^2={\lambda }_{\max }(L{L}^{†})\text{and}{\sigma }_{\min }(L{)}^2={\lambda }_{\min }(L{L}^{†}).$$

$$\left\{A=\left(\begin{array}{cc}\frac{1}{2}& 0\\ 0& 2\end{array}\right),B=\left(\begin{array}{cc}0& -1\\ 1& 0\end{array}\right)\right\}\text{with probability}\{p,1-p\}\text{for }p\in [0,1].$$

$$\displaystyle\gamma_{1}=\left\{\begin{array}[]{l l}\log 2&\text{if }p=1\\ 0&\text{if }p\in(0,1)\end{array}\right.\qquad\text{and}\qquad\gamma_{2}=\left\{\begin{array}[]{l l}\log\frac{1}{2}&\text{if }p=1\\ 0&\text{if }p\in(0,1)\,.\end{array}\right.$$

$$2{\gamma }_{1,n}\le \frac{1}{n}\mathbb{E}\log \mathrm{tr}{L}_1{L}_1^{†}+\frac{1}{n}\sum _{i=2}^n\underset{X\in {\mathbb{X}}_{\mathcal{S}}}{\max }\mathbb{E}\log \mathrm{tr}X{L}_i{L}_i^{†},$$

$$2{\gamma }_{d,n}\ge \frac{1}{n}\sum _{i=1}^n\underset{X\in {\mathbb{X}}_{\mathcal{G}}}{\min }\mathbb{E}\log \mathrm{tr}X{L}_i{L}_i^{†}-\frac{\log d}{n}.$$

$$\displaystyle 2\gamma_{1,n}\leq\frac{1}{n}\mathbb{E}\log\mathrm{tr}\,L_{1}L_{1}^{\dagger}+\frac{1}{n}\sum_{i=2}^{n}\max_{X\in\mathbb{X}_{\mathcal{S}},\ell_{1},\ldots,\ell_{i-1}\in\mathcal{S}}\mathbb{E}\big{[}\log\mathrm{tr}\,XL_{i}L_{i}^{\dagger}|L_{1}=\ell_{1},\ldots,L_{i-1}=\ell_{i-1}\big{]}\,,$$

$$\displaystyle 2\gamma_{d,n}\geq\frac{1}{n}\sum_{i=1}^{n}\min_{X\in\mathbb{X}_{\mathcal{G}},\ell_{1},\ldots,\ell_{i-1}\in\mathcal{G}}\mathbb{E}\big{[}\log\mathrm{tr}\,XL_{i}L_{i}^{\dagger}|L_{1}=\ell_{1},\ldots,L_{i-1}=\ell_{i-1}\big{]}-\frac{\log d}{n}\,.$$

$$\displaystyle 2\gamma_{1,n}\leq\frac{1}{n}\mathbb{E}\log\mathrm{tr}\,L_{1}L_{1}^{\dagger}+\frac{1}{n}\sum_{i=2}^{n}\max_{X\in\mathbb{X}_{\mathcal{S}},\ell_{i-1}\in\mathcal{S}}\mathbb{E}\big{[}\log\mathrm{tr}\,XL_{i}L_{i}^{\dagger}|L_{i-1}=\ell_{i-1}\big{]}$$

$$\displaystyle 2\gamma_{d,n}\geq\frac{1}{n}\sum_{i=1}^{n}\min_{X\in\mathbb{X}_{\mathcal{G}},\ell_{i-1}\in\mathcal{G}}\mathbb{E}\big{[}\log\mathrm{tr}\,XL_{i}L_{i}^{\dagger}|L_{i-1}=\ell_{i-1}\big{]}-\frac{\log d}{n}\,.$$

$$2{\gamma }_1\le \underset{X\in {\mathbb{X}}_{\mathcal{S}}}{\max }\mathbb{E}\log \mathrm{tr}X{L}_1{L}_1^{†},$$

$$2{\gamma }_d\ge \underset{X\in {\mathbb{X}}_{\mathcal{G}}}{\min }\mathbb{E}\log \mathrm{tr}X{L}_1{L}_1^{†}.$$

$$2{\gamma }_1\le \underset{X\in {\mathbb{X}}_{{\mathbb{C}}^{d\times d}}^{\prime }}{\max }\mathbb{E}\log \mathrm{tr}X{L}_1{L}_1^{†}\text{and}2{\gamma }_d\ge \underset{X\in {\mathbb{X}}_{{\mathbb{C}}^{d\times d}}^{\prime }}{\min }\mathbb{E}\log \mathrm{tr}X{L}_1{L}_1^{†},$$

$$2{\gamma }_1\le \underset{X\in {\mathbb{X}}_{{\mathbb{C}}^{d\times d}}}{\max }\mathbb{E}\log \mathrm{tr}X{L}_1{L}_1^{†}\le \log \underset{X\in {\mathbb{X}}_{{\mathbb{C}}^{d\times d}}}{\max }\mathbb{E}\mathrm{tr}X{L}_1{L}_1^{†},$$

$${\mathbb{X}}_{{\mathbb{C}}^{d\times d}}=\{Y^{†}Y/\mathrm{tr}{Y}^{†}Y:Y\in {\mathbb{C}}^{d\times d}\}=\{Y\in {\mathbb{C}}^{d\times d}:Y\ge 0,\mathrm{tr}Y=1\}.$$

$$\underset{X\in {\mathbb{X}}_{{\mathbb{C}}^{d\times d}}}{\max }\mathbb{E}\log \mathrm{tr}X{L}_1{L}_1^{†}\le \log \underset{X\in {\mathbb{X}}_{{\mathbb{C}}^{d\times d}}}{\max }\mathbb{E}\mathrm{tr}X{L}_1{L}_1^{†}.$$

$$2{\gamma }_{1,n}\le \frac{1}{n}\sum _{i=1}^n\mathbb{E}\log {\lambda }_{\max }(L_i{L}_i^{†})\text{and}2{\gamma }_{d,n}\ge \frac{1}{n}\sum _{i=1}^n\mathbb{E}\log {\lambda }_{\min }(L_i{L}_i^{†}),$$

$$\displaystyle 2\gamma_{1,n}=\frac{2}{n}\mathbb{E}\log\sigma_{\max}\Big{(}\prod_{i=1}^{n}L_{i}\Big{)}\leq\frac{2}{n}\mathbb{E}\sum_{i=1}^{n}\log\sigma_{\max}(L_{i})=\frac{1}{n}\sum_{i=1}^{n}\mathbb{E}\log\lambda_{\max}(L_{i}L_{i}^{\dagger})\,,$$

$$\displaystyle 2\gamma_{d,n}=\frac{2}{n}\mathbb{E}\log\sigma_{\min}\Big{(}\prod_{i=1}^{n}L_{i}\Big{)}\geq\frac{2}{n}\mathbb{E}\sum_{i=1}^{n}\log\sigma_{\min}(L_{i})=\frac{1}{n}\sum_{i=1}^{n}\mathbb{E}\log\lambda_{\min}(L_{1}L_{1}^{\dagger})\,,$$

$$2{\gamma }_{1,n}\le \frac{1}{n}\sum _{i=1}^n\mathbb{E}\underset{X\in {\mathbb{X}}_{{\mathbb{C}}^{d\times d}}}{\max }\log \mathrm{tr}X{L}_i{L}_i^{†}\text{and}2{\gamma }_{d,n}\ge \frac{1}{n}\sum _{i=1}^n\mathbb{E}\underset{X\in {\mathbb{X}}_{{\mathbb{C}}^{d\times d}}}{\min }\log \mathrm{tr}X{L}_i{L}_i^{†}.$$

Full text

1 Introduction

Large products of random matrices arise in many areas of theoretical physics. Arguably the most important characterization of such products is given by the Lyapunov exponents that describe the asymptotic behavior of the singular values. Oftentimes one encounters products that do not range over infinitely many matrices. This justifies the definition of non-asymptotic Lyapunov exponents that in the limit converge to the traditional Lyapunov exponents.

More precisely, let $n\in\mathbb{N}$ and $(L_{i})_{i\in[n]}$ be a sequence of random matrices on $\mathbb{C}^{d\times d}$ such that

[TABLE]

where $\sigma_{\max}$ denotes the largest singular value and $\log^{+}t:=\max\{\log t,0\}$. In this manuscript we consider sequences of random matrices only where assumption (1) holds. We define the non-asymptotic Lyapunov spectrum of $(L_{i})_{i\in[n]}$, consisting of the non-asymptotic Lyapunov exponents, as

[TABLE]

where $\sigma_{k}(\cdot)$ denotes the $k$-th singular value. We assume that the singular values are enumerated in decreasing order, i.e., $\sigma_{\max}=\sigma_{1}\geq\sigma_{2}\geq\ldots\geq\sigma_{d}=\sigma_{\min}$.111In case two singular values are equal they are still assigned a different index. Hence, the non-asymptotic Lyapunov spectrum is also ordered in the sense that $\infty>\gamma_{1,n}\geq\gamma_{2,n}\geq\ldots\geq\gamma_{d,n}\geq-\infty$ for all $n\in\mathbb{N}$. The expectation in (2) exists as the largest singular value is submultiplicative [6] and by assumption (1), but it can be $-\infty$.

For many applications it is natural to consider the traditional Lyapunov spectrum which is obtained by taking the limit $n\to\infty$. To ensure that the limit exists we need to impose further assumptions on the random matrices. Let $(L_{i})_{i\in\mathbb{N}}$ be a stationary sequence of random matrices that satisfies (1). Then the (asymptotic) Lyapunov spectrum, consisting of the (asymptotic) Lyapunov exponents, is given by

[TABLE]

A precise argument of the well-known fact that the limit in (3) exists is given in Appendix A for the sake of completeness.

If we impose another assumption on the random matrices, i.e., that they are stationary and ergodic, it can be shown that $\mathbb{P}$-almost surely the expectation in the definition of Lyapunov spectrum can be dropped, where $\mathbb{P}$ is the probability measure of the underlying probability space the random matrices are defined on. More precisely, if $(L_{i})_{i\in\mathbb{N}}$ is a stationary ergodic sequence of random matrices on $\mathbb{C}^{d\times d}$ that satisfies (1) then

[TABLE]

The justification for this is due to Kingman’s subadditive ergodic theorem [30] (see also the two celebrated and historically older theorems by Oseledets [38] and Kesten-Furstenberg [24, Theorem 2] which can be derived from Kingman’s result).

The maximal Lyapunov exponent $\gamma_{1}$ has a dominant role within the Lyapunov spectrum. Its definition can be rewritten in terms of Schatten norms as

[TABLE]

where \left\lVert L\right\rVert_{p}:=\big{(}\mathrm{tr}|L|^{p}\big{)}^{\frac{1}{p}} and $|L|:=\sqrt{L^{\dagger}L}$. In the limit $p\to\infty$ we recover the spectral norm that is equal to the largest singular value denoted by $\sigma_{\max}(L)$ or $\sigma_{1}(L)$. The maximal Lyapunov exponent defined in (5) is independent of the parameter $p$ since all Schatten $p$-norms are equivalent [6] and $d<\infty$.

The minimal Lyapunov exponent $\gamma_{d}$ is a meaningful quantity only for distributions on the group of invertible matrices on $\mathbb{C}^{d\times d}$, denoted by $\mathrm{GL}(d,\mathbb{C})$. This follows from the fact that whenever a probability distribution has positive weight on a matrix that is not invertible we have $\gamma_{d}=-\infty$, which is obvious from the definition of $\gamma_{d}$.

The Lyapunov spectrum and in particular the maximal Lyapunov exponent plays a crucial role in several different areas of physics and mathematics. Arguably its most prominent applications are in the study of dynamical systems and of disordered materials. A positive maximal Lyapunov exponent for a sequence of random matrices describing the discretized time evolution of a dynamical system for example indicates that the system is chaotic. More precisely, the maximal Lyapunov exponent characterizes the sensitivity to initial conditions of a system. $\gamma_{1}$ is proportional to the inverse time rate at which two nearby trajectories diverge [9]. In the study of an Ising model, the maximal Lyapunov exponent is directly related to the free energy of the system and to the rate of the correlation decay [14]. In the Schrödinger equation with a random potential Lyapunov coefficients yield direct bounds on the localization length of the wave function [10, 1]. In information theory, the entropy rate of hidden Markov processes is directly related to the maximal Lyapunov exponent for a sequence of random matrices describing the Markov process [18]. Many more problems have been reduced to the study of the Lyapunov spectrum. We refer the interested reader to [14, 1] for more details.

Computing the Lyapunov spectrum and in particular the minimal and maximal Lyapunov exponents turns out to be challenging. There is no explicit formula known that can be evaluated easily.222Kingman mentions in [30]: “Pride of place among the unsolved problems of subadditive ergodic theory must go to the calculation of the constant $\gamma_{1}$.” An exception is the commutative case discussed in Section 4.1, i.e., if the random matrices commute pairwise. There are also a few specific sequences of $2\times 2$ matrices such that the corresponding maximal Lyapunov exponent can be computed (see, e.g., [33, 34, 12, 11] and references therein). For arbitrary finite dimensions the maximal Lyapunov exponent is only known analytically for the case of independent and identically distributed (i.i.d.) Gaussian matrices [36], i.e., matrices with entries that are independent standard Gaussians, or small variations thereof [22, 28]. Furstenberg proved a powerful lower bound for the maximal Lyapunov exponent that plays an important role in the mathematical description of disordered materials. He showed that for matrices belonging to the special linear group $\mathrm{SL}(d,\mathbb{R})$, i.e., the group of matrices over $\mathbb{R}^{d\times d}$ with determinant equal to $1$, the maximal Lyapunov exponent is strictly positive for most distributions on this group [23].

It has been shown that the maximal Lyapunov exponent cannot be approximated by an algorithm in full generality [41, 42]. Under the assumption that the random matrices have nonnegative entries the problem is easier to deal with. For example certain limit type statements have been derived [25, 35]. Furthermore, for random matrices with nonnegative entries there exist approximation algorithms for the maximal Lyapunov exponent that do converge [39, 40], see also references therein for different algorithms with a slower rate of convergence.

Result.

We prove that for any sequence $(L_{i})_{i\in\mathbb{N}}$ of i.i.d. random matrices on a semigroup $\mathcal{S}\leq\mathbb{C}^{d\times d}$, or on a group $\mathcal{G}\leq\mathrm{GL}(d,\mathbb{C})$, that satisfies (1) we have

[TABLE]

where $\mathbb{X}_{\mathcal{S}}:=\{Y^{\dagger}Y/\mathrm{tr}\,Y^{\dagger}Y:Y\in\mathcal{S}\}$. If we drop the (semi)group structure the optimizer can be assumed to be rank-one, i.e.,

[TABLE]

where $\mathbb{X}^{\prime}_{\mathbb{C}^{d\times d}}:=\{Y^{\dagger}Y/\mathrm{tr}\,Y^{\dagger}Y:Y\in\mathbb{C}^{d\times d},\,\mathrm{rank}\,Y=1\}$. We refer to Theorem 3.1 for a more precise and more general result that provides upper and lower bounds for the non-asymptotic maximal and minimal Lyapunov exponents $\gamma_{1,n}$ and $\gamma_{d,n}$, respectively. The asymptotic statements are given by Corollaries 3.3 and 3.4. The bounds are tight for independent and identically distributed diagonal matrices. This is discussed in Section 4.1. Furthermore, Sections 4.2 and 4.3 present various scenarios where the bounds are either tight or outperform known bounds on the maximal or minimal Lyapunov exponent.

The bounds (6) are closely related with an information theoretic tool called entropy accumulation theorem (EAT) [16, 15]. The EAT ensures that the operationally relevant entropic quantities of a multiparty system (called smooth min-and max-entropies [31]) can be bounded by the sum of the von Neumann entropies of its individual parts viewed on a worst case scenario. In Section 5.1 we explain this connection. In particular we show that the bounds (6) have been motivated by the EAT and suggest an accumulation theorem for the relative entropy.

Efficient evaluation of the bounds.

One crucial difference between the bounds in (6) for the maximal and minimal Lyapunov exponents and their definition (3) is that the former are given by formulas that contain a single random matrix only and hence do not include limits of products of infinitely many random matrices. Depending on the structure of $\mathbb{X}_{\mathcal{S}}$ and $\mathbb{X}_{\mathcal{G}}$ the maximization and minimization in (6) may still not be simple to evaluate. To circumvent this problem we can relax $\mathbb{X}_{\mathcal{S}}$ and $\mathbb{X}_{\mathcal{G}}$ to $\mathbb{X}_{\mathbb{C}^{d\times d}}$ which weakens the bounds but makes them easier to compute. Proposition 3.6 shows that the bound (6) on the maximal Lyapunov exponent for $\mathbb{X}_{\mathbb{C}^{d\times d}}$ can be evaluated efficiently via convex programming [8].

Structure.

Section 2 introduces the notation, reviews basic properties of eigenvalues of Hermitian matrices, and summarizes known results on the continuity of the maximal Lyapunov exponent. In Section 3 we present and prove the main result and discuss its implications. Section 4 presents various examples that illustrate how to use the bounds of the main result in practice and give insights about their performance. Finally in Section 5 we discuss two connections between Lyapunov exponents and entropy and explain why the bounds on $\gamma_{1}$ and $\gamma_{d}$ can be useful in this context.

2 Preliminaries

2.1 Notation

For $n\in\mathbb{N}$ let $[n]:=\{1,2,\ldots,n\}$. We denote the Löwner partial order on positive semidefinite matrices by $\geq$, i.e., $X\geq 0$ states that $X$ is a positive semidefinite matrix. For a matrix $L$ we write $L^{\dagger}$ for its conjugate transpose. The natural logarithm is denoted by $\log(\cdot)$. The general and special linear group over $\mathbb{C}^{d\times d}$ are denoted by $\mathrm{GL}(d,\mathbb{C})$ and $\mathrm{SL}(d,\mathbb{C})$, respectively. If $\mathcal{G}$ is a subgroup of $\mathcal{H}$ we write $\mathcal{G}\leq\mathcal{H}$. For a semigroup $\mathcal{S}\leq\mathbb{C}^{d\times d}$ we define the following set

[TABLE]

If there exists $Y\in\mathcal{S}$ such that $Y^{\dagger}Y=0$ then we define $Y^{\dagger}Y/\mathrm{tr}\,Y^{\dagger}Y=0$, i.e., $\mathbb{X}_{\mathcal{S}}$ contains the zero matrix.333We note that this cannot happen if $\mathcal{S}$ forms a group and hence is only relevant in case $\mathcal{S}$ is a semigroup. We further define the set

[TABLE]

Let $(L_{n})_{n\in\mathbb{N}}$ be a sequence of random matrices on $\mathbb{C}^{d\times d}$ with $(\Omega,\mathcal{F},\mathbb{P})$ the associated probability space. Let $T:\Omega\to\Omega$ be a shift operator that drops the first coordinate and shifts the others one place to the left. An event $\mathcal{A}\in\mathcal{F}$ is said to be shift invariant if $\mathcal{A}=T^{-1}\mathcal{A}$. $(L_{n})_{n\in\mathbb{N}}$ is called stationary if for every $k\geq 1$ it has the same distribution as the shifted sequence $(L_{k+n})_{n\in\mathbb{N}}$, i.e., for each $m$, $(L_{1},\ldots,L_{m})$ and $(L_{k},...,L_{k+m})$ have the same distribution. $(L_{n})_{n\in\mathbb{N}}$ is called ergodic if every shift invariant event $\mathcal{A}$ is trivial, i.e., $\mathbb{P}(\mathcal{A})\in\{0,1\}$. We note that it is a simple exercise to show that i.i.d. sequences are stationary and ergodic [17].

2.2 Variational formulas for eigenvalues

For any matrix $L\in\mathbb{C}^{d\times d}$ we have $LL^{\dagger}\geq 0$ as $\langle z^{\dagger}L,L^{\dagger}z\rangle=\left\lVert L^{\dagger}z\right\rVert^{2}\geq 0$ for all $z\in\mathbb{C}^{d}$. The Cholesky decomposition ensures that every positive semidefinite matrix $0\leq A\in\mathbb{C}^{d\times d}$ can be written as $LL^{\dagger}$ for some $L\in\mathbb{C}^{d\times d}$ [5, Fact 8.9.37].444More precisely we can assume that $L$ is a lower triangular matrix with nonnegative diagonal entries. The largest and smallest eigenvalue of $LL^{\dagger}$ are denoted by $\lambda_{\max}(LL^{\dagger})$ and $\lambda_{\min}(LL^{\dagger})$, respectively. These eigenvalues can be expressed as a semidefinite program. Using the notation (8) we have

[TABLE]

For $L\in\mathcal{G}\leq\mathrm{GL}(d,\mathbb{C})$ the following relation holds

[TABLE]

This can be seen as for $Y=L^{-1}\in\mathcal{G}$ the right-hand side simplifies to $(\mathrm{tr}\,(LL^{\dagger})^{-1})^{-1}=\|(LL^{\dagger})^{-1}\|_{1}^{-1}\leq\lambda_{\max}((LL^{\dagger})^{-1})^{-1}=\lambda_{\min}(LL^{\dagger})$. For $L\in\mathbb{C}^{d\times d}$ we denote its singular values by $\sigma_{\max}(L)=\sigma_{1}(L)\geq\sigma_{2}(L)\geq\ldots\geq\sigma_{d}(L)=\sigma_{\min}(L)$. The largest and smallest singular value are related to the largest and smallest eigenvalues [5, Fact 9.13.1] by

[TABLE]

2.3 Continuity of the maximal Lyapunov exponent

Consider a model where a fixed invertible matrix $L_{i}$ is chosen with probability $p_{i}>0$. The following theorem shows that the corresponding maximal Lyapunov exponent is continuous in $p_{i}$ and $L_{i}$.

Theorem 2.1** ([3]).**

The function $(p_{i},L_{i})\mapsto\gamma_{1}$ is continuous for $p_{i}>0$ where $p_{i}$ are probabilities and $L_{i}$ invertible matrices.

Because the reference [3] is in preparation we present a proof for the assertion of Theorem 2.1 in Appendix B, assuming the version of the theorem explicitly stated in [44, Theorem 3.5]. The following example discussed in [29] shows that the continuity may break down if some of the probabilities vanish.

Example 2.2**.**

Let $(L_{i})_{i\in\mathbb{N}}$ be i.i.d. matrices over

[TABLE]

It is straightforward to see that

[TABLE]

We note that this discontinuity of $\gamma_{1}$ is not in contradiction with the statement above since $\mu=p\delta_{A}+(1-p)\delta_{B}$ does not converge to $\delta_{A}$ in the $\mathcal{T}$ topology when $p\to 1$, because $\mathrm{supp}\,\mu=\{A,B\}$ does not converge to $\mathrm{supp}\,\delta=\{A\}$ in the Hausdorff topology.

3 Main results and proofs

We next state the main result which is an upper and lower bound for the non-asymptotic maximal and minimal Lyapunov exponents, respectively.

Theorem 3.1**.**

Let $d,n\in\mathbb{N}$ and let $(L_{i})_{i\in[n]}$ be a sequence of independent random matrices on a semigroup $\mathcal{S}\leq\mathbb{C}^{d\times d}$ that satisfies (1). Then

[TABLE]

where $\mathbb{X}_{\mathcal{S}}=\{Y^{\dagger}Y/\mathrm{tr}\,Y^{\dagger}Y:Y\in\mathcal{S}\}$. If $(L_{i})_{i\in[n]}$ is distributed on a group $\mathcal{G}\leq\mathrm{GL}(d,\mathbb{C})$ we further have

[TABLE]

The proof of Theorem 3.1 is given in Section 3.1 below.

Remark 3.2** (Dependent random matrices with a discrete probability distribution.).**

In case of discrete probability distributions on $\mathcal{S}\leq\mathbb{C}^{d\times d}$ we can prove a version of Theorem 3.1 without the independence assumption of the random matrices, i.e,

[TABLE]

where $\mathbb{X}_{\mathcal{S}}=\{Y^{\dagger}Y/\mathrm{tr}\,Y^{\dagger}Y:Y\in\mathcal{S}\}$. If $(L_{i})_{i\in[n]}$ is distributed on a group $\mathcal{G}\leq\mathrm{GL}(d,\mathbb{C})$ we further have

[TABLE]

We note that the proof follows the same lines as the proof of Theorem 3.1 given in Section 3.1. The bounds (21) and (22) can be simplified if the random matrices satisfy a certain dependence structure. For example in case $(L_{i})_{i\in[n]}$ form a Markov chain in order $L_{k-1}\leftrightarrow L_{k}\leftrightarrow L_{k+1}$ the formulas (21) and (22) simplify to

[TABLE]

and

[TABLE]

If the random matrices form a Markov chain or are even i.i.d. the bounds can be further simplified in the limit $n\to\infty$.

Corollary 3.3**.**

Let $d\in\mathbb{N}$ and let $(L_{i})_{i\in\mathbb{N}}$ be a sequence of i.i.d. random matrices on a semigroup $\mathcal{S}\leq\mathbb{C}^{d\times d}$ that satisfies (1). Then

[TABLE]

where $\mathbb{X}_{\mathcal{S}}=\{Y^{\dagger}Y/\mathrm{tr}\,Y^{\dagger}Y:Y\in\mathcal{S}\}$. If $(L_{i})_{i\in\mathbb{N}}$ is distributed on a group $\mathcal{G}\leq\mathrm{GL}(d,\mathbb{C})$ we further have

[TABLE]

Proof.

The assertion follows from Theorem 3.1 by considering the limit $n\to\infty$. The justification that the limits do exist for i.i.d. random matrices is given in Section 1. We note that assumption (1) implies that $\mathbb{E}\log\mathrm{tr}L_{1}L_{1}^{\dagger}\leq d\,\mathbb{E}\log\lambda_{\max}(L_{1}L_{1}^{\dagger})<\infty$ and hence $\lim_{n\to\infty}\frac{1}{n}\mathbb{E}\log\mathrm{tr}\,L_{1}L_{1}^{\dagger}=0$ which shows that the first term in (19) vanishes in the limit $n\to\infty$. ∎

There is another asymptotic version of Theorem 3.1 where we enforce a rank-one constraint on the optimizers.

Corollary 3.4**.**

Let $d\in\mathbb{N}$ and let $(L_{i})_{i\in\mathbb{N}}$ be a sequence of i.i.d. random matrices on $\mathrm{GL}(d,\mathbb{C})$ with a distribution that satisfies (1) and has compact support. Then

[TABLE]

where $\mathbb{X}^{\prime}_{\mathbb{C}^{d\times d}}=\{Y^{\dagger}Y/\mathrm{tr}\,Y^{\dagger}Y:Y\in\mathbb{C}^{d\times d},\,\mathrm{rank}\,Y=1\}$.

The proof of Corollary 3.4 is given in Section 3.2. We note that the major difference between Corollary 3.3 and Corollary 3.4 is that in the latter we can assume that the optimizer has rank-one, at the cost of considering $\mathbb{C}^{d\times d}$ instead of $\mathcal{S}$ or $\mathcal{G}$. Example 4.5 shows that there exist scenarios where Corollary 3.4 outperforms Corollary 3.3. Example 4.5 presents a case where the bounds from Corollary 3.4 are even tight.

Remark 3.5**.**

We note that Corollary 3.4 unlike Corollary 3.3 does not respect the possible (semi)group structure of the random matrices. As visible from the proof it is possible to strengthen Corollary 3.4. More precisely suppose we have a sequence of random matrices $(L_{i})_{i\in\mathbb{N}}$ on a group $\mathcal{G}\leq\mathbb{C}^{d\times d}$ with distribution $\mu$ such that there exits a family of joint distributions $(\mu^{\prime}_{n})_{n\in\mathbb{N}}$ such that

$\lim_{n\to\infty}\left\lVert\mu^{\prime}_{n}-\mu\right\rVert_{1}=0$ 2. 2.

$\gamma_{1,n}^{\prime}>\gamma_{2,n}^{\prime}$ for all $n\in\mathbb{N}$ where $\gamma_{1,n}^{\prime}$ and $\gamma_{2,n}^{\prime}$ denote the largest and second largest Lyapunov exponents of $(L^{\prime}_{i,n})_{i\in\mathbb{N}}$ distributed according to $\mu^{\prime}_{n}$ 3. 3.

for all $G\in\mathcal{G}$ we have $\Pi_{GG^{\dagger}}\in\mathcal{G}$, where $\Pi_{GG^{\dagger}}$ denotes the projector onto the eigenspace corresponding to the largest eigenvalue of $GG^{\dagger}$.

Then it follows from the proof of Corollary 3.4 that (27) is valid for $\mathbb{X}^{\prime}_{\mathcal{G}}$.

As an example, it is an easy exercise to verify that the group $\mathcal{G}=\mathcal{D}(d,\mathbb{C})$ of commutative diagonalizable invertible matrices satisfies Properties 1, 2, and 3 above with $\mu^{\prime}_{n}=(1-\frac{1}{n})\mu+\frac{1}{n}\tilde{\mu}$ where $\tilde{\mu}$ is the joint distribution of uniformly distributed matrices on $\mathcal{D}(d,\mathbb{C})$. We refer to Section 4.1 for a more precise discussion of the commutative case.

Evaluating the bounds in Corollary 3.3 above may not be straightforward as the sets $\mathbb{X}_{\mathcal{S}}$ and $\mathbb{X}_{\mathcal{G}}$ can be complicated depending on the structure of $\mathcal{S}$ and $\mathcal{G}$. It is always possible to weaken the bounds by relaxing $\mathbb{X}_{\mathcal{S}}$ and $\mathbb{X}_{\mathcal{G}}$ to $\mathbb{X}_{\mathbb{C}^{d\times d}}$ which is equal to the set of positive semidefinite $d\times d$ matrices with trace one, also known as density matrices.555We note that in case $\mathbb{X}_{\mathcal{S}}$ contains the zero matrix this matrix can be removed from the set because for the upper bound the zero matrix would only be relevant in trivial scenarios where the upper bound is $-\infty$. One important advantage of working with $\mathbb{X}_{\mathbb{C}^{d\times d}}$ is that due to the convexity of $\mathbb{X}_{\mathbb{C}^{d\times d}}$ the bound for $\gamma_{1}$ can be efficiently evaluated, i.e., the maximization is efficiently computable.

Proposition 3.6**.**

If the bound in (25) is relaxed by using $\mathbb{X}_{\mathbb{C}^{d\times d}}$ instead of $\mathbb{X}_{\mathcal{S}}$ it can be computed efficiently. More precisely, we have

[TABLE]

where the first and second bounds are a convex and semidefinite optimization problem, respectively.

Proof.

We start by recalling that the set $\mathbb{X}_{\mathbb{C}^{d\times d}}$ can be written as

[TABLE]

Jensen’s inequality together with the monotonicity of the logarithm implies that

[TABLE]

The first expression is a convex optimization problem as we are maximizing a concave function over a convex set [8]. The second expression is even a semidefinite program as the objective function is linear. ∎

All semidefinite programs and most convex optimization problems can be solved efficiently by modern algorithms [8].

The following remark compares the bounds from Theorem 3.1 with bounds that can be obtained straightforwardly by using the submultiplicativity of the largest singular value.

Remark 3.7** (Comparison with trivial bounds).**

The submultiplicativity of the largest singular value, i.e., $\sigma_{\max}(L_{1}L_{2})\leq\sigma_{\max}(L_{1})\sigma_{\max}(L_{2})$ for any two matrices $L_{1}$ and $L_{2}$ [6], implies the following non-asymptotic bounds for the maximal and minimal Lyapunov exponent

[TABLE]

where for the lower bound we assume that $(L_{n})_{n\in\mathbb{N}}$ is such that only invertible matrices have positive probability to occur (as otherwise $\gamma_{d}=-\infty$). This is correct since

[TABLE]

where the final step uses (12). The lower bound follows by similar arguments. By the submultiplicativity of the maximal singular value we have $\sigma_{\min}(L_{1}L_{2})\geq\sigma_{\min}(L_{1})\sigma_{\min}(L_{2})$ for any $L_{1},L_{2}\in\mathrm{GL}(d,\mathbb{C})$.666This follows from that fact that for any invertible matrix $L$ we have $\sigma_{\max}(L)=1/\sigma_{\min}(L^{-1})$. Hence we obtain

[TABLE]

where the final step follows from (12).

If the random matrices are independent the bounds (31) immediately follow from (19) and (20). To see this we rewrite (31) using the variational formulas (10) as

[TABLE]

This shows that the bounds (19) and (20) are stronger than (31) and (34) as the former imply the latter by relaxing them (by using $\mathbb{X}_{\mathbb{C}^{d\times d}}$ instead of $\mathbb{X}_{\mathcal{S}}$ and $\mathbb{X}_{\mathcal{G}}$) and swapping the expectation with the maximization and minimization, respectively.777We note that in the lower bound for $\gamma_{d,n}$ in (20) we have an additional term $O(\frac{\log d}{n})$ term, which however in practice does not matter as it vanishes for large values of $n$. We note that in case of dependent random matrices the bounds (31) may outperform (19) and (20).

It is possible to distill further bounds on $\gamma_{1}$ and $\gamma_{d}$ from the bounds given by Theorem 3.1 via a simple relation between the largest and smallest singular value of an invertible matrix, i.e., $\sigma_{\max}(L)=1/\sigma_{\min}(L^{-1})$.

Remark 3.8**.**

Let $(L_{i})_{i\in\mathbb{N}}$ be a sequence of i.i.d. random matrices on a group $\mathcal{G}\leq\mathrm{GL}(d,\mathbb{C})$. By utilizing the fact that $\sigma_{\max}(L)=1/\sigma_{\min}(L^{-1})$ we find

[TABLE]

and an analogous expression for $\gamma_{d}$. We can apply this simple observation to the bounds from Corollary 3.3 and find

[TABLE]

and

[TABLE]

We note that instead of Corollary 3.3 we could also improve the bounds of Theorem 3.1 and Corollary 3.4 with this observation. We note that in case the group $\mathcal{G}$ has additional symmetry this may be useful to further improve the bounds. For example in case of $\mathcal{G}=\mathrm{SL}(2,\mathbb{R})$ we know that $\gamma_{1}=-\gamma_{2}$ which turns out to be useful. This is explained in Example 4.2.

As mentioned in Section 1 for various applications it is of interest to show that the top Lyapunov exponent is strictly positive. Hence a good lower bound on $\gamma_{1}$ would be a desirable tool. The techniques used in this manuscript lead to an upper bound for $\gamma_{1}$ and a lower bound on $\gamma_{d}$, but presumably not to a good lower bound on $\gamma_{1}$.888A lower bound for $\gamma_{d}$ is by definition also a lower bound for $\gamma_{1}$, which however is not necessarily good since $\gamma_{d}$ and $\gamma_{1}$ can be far apart. The interested reader may consult [32] for recent advances on proving analytical lower bounds on $\gamma_{1}$.

We conclude this section with a remark about a generic possibility to further improve the bounds. The idea is to apply the new bounds to (small) products of matrices, i.e., for example we could consider a sequence of matrices $(L_{1,i}L_{2,i})_{i\in\mathbb{N}}$. The larger we choose these products, the better the bound performs at the cost that the evaluation of the bounds gets more complicated.

3.1 Proof of Theorem 3.1

To simplify notation we denote $L^{n}:=\prod_{k=1}^{n}L_{k}$ and $L_{m}^{n}:=\prod_{k=m}^{n}L_{k}$ for $m\leq n$. Let $\mu_{L^{n}}$ denote the joint distribution of $L^{n}$. We first prove the upper bound for the maximal Lyapunov exponent. By properties of Schatten norms, i.e., $\sigma_{\max}(A)\leq\left\lVert A\right\rVert_{2}$ for every $A\in\mathbb{C}^{d\times d}$ we have

[TABLE]

and thus

[TABLE]

We can write

[TABLE]

where $\mu_{(L^{i-1},L_{i+1}^{n})}(\mathrm{d}L^{i-1},\mathrm{d}L_{i+1}^{n},L_{i})$ denotes the conditional joint distribution of $(L^{i-1},L_{i+1}^{n})$. We now have a natural upper bound on each term $i\geq 2$

[TABLE]

We note that for $i\geq 2$ we have

[TABLE]

which thus justifies the inequality above. Combining the previous steps gives

[TABLE]

With a similar proof technique we obtain the asserted lower bound for the minimal Lyapunov exponent. By assumption $\mu$ is defined on $\mathcal{G}\leq\mathrm{GL}(d,\mathbb{C})$. With the inequality for the smallest eigenvalue given in (11) we have

[TABLE]

Using the same notation as above we find

[TABLE]

Let us denote the optimizer in the minimization above by $\bar{W}$.999Without loss of generality we can assume that $\mathrm{tr}\bar{W}^{\dagger}\bar{W}=1$ since we can renormalize the terms as we wish. With some abuse of notation, i.e., $L^{0}=\mathrm{id}_{d}$ we can write

[TABLE]

Each term for $i\geq 1$ can be bounded from below as

[TABLE]

The above inequality holds because

[TABLE]

which thus justifies the inequality above. Combining the previous steps gives

[TABLE]

which completes the proof. ∎

3.2 Proof of Corollary 3.4

We note that the major difference between Theorem 3.1 and Corollary 3.4 is that in the asymptotic setting of Corollary 3.4 we can assume that the optimizer is rank-one. Without this rank-one constraint the result would follow immediately from Theorem 3.1 by considering the limit $n\to\infty$ which is explained in detail in the proof of Corollary 3.3. The justification that the limits do exist for stationary random matrices is given in Section 1.

It thus remains to prove why we can add the rank-one constraint to the optimizers. To see this recall that following the proof of Theorem 3.1 we find

[TABLE]

From Kingman’s subadditive ergodic theorem, see (4), we find that for any $\delta>0$ there is an $\varepsilon_{n}$ with $\lim_{n\to\infty}\varepsilon_{n}=0$ such that

[TABLE]

and hence

[TABLE]

To simplify notation let

[TABLE]

Consider the eigendecomposition $M_{n}=\sum_{k=1}^{d}\lambda_{k}^{(n)}\Pi_{k}^{(n)}$ where $\lambda_{1}^{(n)}\geq\lambda_{2}^{(n)}\geq\ldots\geq\lambda_{d}^{(n)}$ denote the eigenvalues of $M_{n}$ and $\Pi_{k}^{(n)}$ is the projector onto the eigenspace of $\lambda_{k}^{(n)}$. As a result we find

[TABLE]

We note that $\lambda_{k}(L^{n}(L^{n})^{\dagger})=\lambda_{k}^{(n)}$ since since the eigenvalues of $YY^{\dagger}$ and $Y^{\dagger}Y$ are equal for any $Y\in\mathbb{C}^{d\times d}$. Statement (57) thus ensures that for sufficiently large $n$ we have with high probability

[TABLE]

As a result we find with probability $1-\varepsilon_{n}$

[TABLE]

For the moment we assume that $\gamma_{1}>\gamma_{2}$. Plugging this into (55) shows that

[TABLE]

Recall that by assumption $\mathbb{E}\log\sigma_{\max}(L_{1})\leq\kappa<\infty$. If we split the sum in $i\in\{2,\ldots,\lfloor\sqrt{n}\rfloor\}$ and $i\in\{\lfloor\sqrt{n}\rfloor+1,\ldots,n\}$ we find

[TABLE]

for $\mathbb{X}^{\prime}_{\mathbb{C}^{d\times d}}=\{Y^{\dagger}Y/\mathrm{tr}\,Y^{\dagger}Y:Y\in\mathbb{C}^{d\times d},\,\mathrm{rank}\,Y=1\}$. The second step uses that $\lim_{n\to\infty}\varepsilon_{n}=0$. The penultimate step is true since $\Pi_{1}^{(n)}$ is a rank-one projector for all $n\in\mathbb{N}$.

In case $\gamma_{1}=\gamma_{2}$ we use a continuity argument to prove the assertion. Let $\mu$ denote the joint distribution of the matrices $(L_{i})_{i\in\mathbb{N}}$. We consider a family of joint distributions $(\mu^{\prime}_{n})_{n\in\mathbb{N}}$ on $\mathbb{C}^{d\times d}$ that is a perturbed version of $\mu$ of the form $\mu^{\prime}_{n}=(1-\frac{1}{n})\mu+\frac{1}{n}\tilde{\mu}$ where $\tilde{\mu}$ is the joint distribution of a sequence of matrices on $\mathbb{C}^{d\times d}$ whose entries are chosen uniformly at random with magnitude at most $1$. It is easy to see that $\lim_{n\to\infty}\left\lVert\mu^{\prime}_{n}-\mu\right\rVert_{1}=0$. The Lyapunov spectrum of the sequence $(L^{\prime}_{i,n})_{i\in\mathbb{N}}$ that is distributed according to $\mu^{\prime}_{n}$ is simple (i.e., all Lyapunov exponents are distinct) for all $n\in\mathbb{N}$, i.e., in particular $\gamma_{1}(\mu^{\prime}_{n})>\gamma_{2}(\mu^{\prime}_{n})$ for all $n\in\mathbb{N}$. This follows from [43, Theorem 8.1] together with [43, Exercise 8.1]. Since $\mu\mapsto\gamma_{1}(\mu)$ is continuous for this setup as explained in Section 2.3 [44, Theorem 3.5] we find

[TABLE]

Hölder’s inequality implies that for all $X\in\mathbb{X}^{\prime}$

[TABLE]

Since the matrix $L_{1}$ is invertible $\|\log\mathrm{tr}\,XL_{1}L_{1}^{\dagger}\|_{\infty}=\kappa<\infty$. Together with (71) this gives

[TABLE]

It remains to prove the lower bound of $\gamma_{d}$ stated in Corollary 3.4. We note that $\mathbb{X}^{\prime}_{\mathbb{C}^{d\times d}}$ is the boundary of $\mathbb{X}_{\mathbb{C}^{d\times d}}$. The concavity of the logarithm ensures the minimum is attained at the boundary which thus proves the assertion. ∎

3.3 Comparison of new bounds with existing results

Since a direct calculation of Lyapunov exponents is known to be notoriously difficult, it is natural trying to derive good upper and lower bounds that can be evaluated efficiently. In Section 1 we discussed some existing results such as Furstenberg’s celebrated lower bound on $\gamma_{1}$ for random matrices on the special linear group [23].

In this work (see Corollaries 3.3 and 3.4) we presented (i) an upper bound on $\gamma_{1}$ and (ii) a lower bound on $\gamma_{d}$. To the best of our knowledge the bound (ii) is entirely novel and we are not aware of a comparable existing bound in the literature. The bound (i) can be seen as a considerable refinement of a known result [40, Equation (23)]. The novel bound has two important advantages

The upper bound from Corollary 3.3 can utilize a possible semigroup structure of the random matrices. More precisely, the maximization in (25) is over the set $\mathbb{X}_{\mathcal{S}}$, whereas the existing bound [40, Equation (23)] cannot reflect this structure and hence the maximization has to be done over the set $\mathbb{X}_{\mathbb{C}^{d\times d}}$. We note that $\mathbb{X}_{\mathcal{S}}$ can be considerably smaller than $\mathbb{X}_{\mathbb{C}^{d\times d}}$ which makes the upper bound (25) substantially better than [40, Equation (23)]. This is illustrated by Example 4.3. 2. 2.

The upper bound from Corollary 3.4 is an improvement over [40, Equation (23)] as the maximization is taken over the set $\mathbb{X}^{\prime}_{\mathbb{C}^{d\times d}}$ instead of $\mathbb{X}_{\mathbb{C}^{d\times d}}$. Because $\mathbb{X}^{\prime}_{\mathbb{C}^{d\times d}}$ is smaller than $\mathbb{X}_{\mathbb{C}^{d\times d}}$, as it requires the positive definite unit trace matrices to have rank one, the new bound can be substantially better compared to [40, Equation (23)]. This is illustrated by Example 4.5.

4 Examples

In this section we discuss some examples and show how the bounds from Theorem 3.1 and Corollaries 3.3 and 3.4 perform in practice. We start with a precise analysis of the commutative case.

4.1 The commutative case

Let $\mathcal{D}(d,\mathbb{C})\leq\mathrm{GL}(d,\mathbb{C})$ denote the group of diagonal invertible matrices over $\mathbb{C}^{d\times d}$. For any sequence $(L_{i})_{i\in\mathbb{N}}$ of i.i.d. random matrices on $\mathcal{D}(d,\mathbb{C})$ the maximal and minimal Lyapunov exponent are given by

[TABLE]

where $\lambda_{\max}(\cdot)$ and $\lambda_{\min}(\cdot)$ denote the largest and smallest eigenvalue, respectively. This fact is formally proven in Section 4.1.2 below and was already observed in [14]. Let $\bar{\mathcal{D}}(d,\mathbb{C})$ denote the semigroup of diagonal (not necessarily invertible) matrices on $\mathbb{C}^{d\times d}$. Corollary 3.4 together with Remark 3.5 implies

[TABLE]

for $\mathbb{X}^{\prime}_{\bar{\mathcal{D}}}=\{YY^{\dagger}/\mathrm{tr}\,YY^{\dagger}:Y\in\bar{\mathcal{D}}(d,\mathbb{C}),\mathrm{rank}\,Y=1\}$. As claimed in Section 1 our bounds are tight in the commutative case, i.e.,

[TABLE]

and

[TABLE]

4.1.1 Proof of (76) and (77)

We start by proving (76). To see why this is correct we note that because $\bar{\mathcal{D}}(d,\mathbb{C})$ is the semigroup of commutative diagonalizable matrices there exists a unitary matrix $U$ that diagonalizes $X$ as well as $LL^{\dagger}$ for all $L\sim\mu$, i.e., $X=U\Lambda U^{\dagger}$ for $\Lambda=\mathrm{diag}(\lambda_{1},\ldots,\lambda_{d})$ with $\lambda_{i}=1$ for some $i\in[d]$ and $\lambda_{j}=0$ for $j\neq i$, as well as $LL^{\dagger}=U\Xi U^{\dagger}$ for $\Xi=\mathrm{diag}(\xi_{1},\ldots,\xi_{d})$ with $\xi_{i}\geq 0$. We then find

[TABLE]

The same argumentation shows that

[TABLE]

is also correct. ∎

4.1.2 Proof of (74)

To prove (74) we start with the following simple lemma.

Lemma 4.1**.**

Let $H$ be a Hermitian matrix. Then

[TABLE]

Proof.

Every Hermitian matrix $H$ can be diagonalized, i.e., it can be written as $H=U\Lambda U^{\dagger}$, where $U$ is unitary and $\Lambda$ is a real diagonal matrix containing the eigenvalues of $H$. We thus find

[TABLE]

Analogously we have

[TABLE]

∎

We prove the assertion for the maximal Lyapunov exponent. Since $(L_{i})_{i\in\mathbb{N}}$ are distributed on $\mathcal{D}(d,\mathbb{C})$ we find

[TABLE]

where the final step uses the Kesten-Furstenberg result. Lemma 4.1 and the fact that $(L_{i})_{i\in\mathbb{N}}$ are distributed on $\mathcal{D}(d,\mathbb{C})$ gives $\mathbb{P}$-a.s.

[TABLE]

where the penultimate step uses the continuity of the largest eigenvalue. The final step follows from the law of large numbers [17]. The statement for the minimal Lyapunov exponent follows by the same line of arguments ∎

4.2 The special linear group

In this section we discuss one example that is of particular relevance in physics. The Lyapunov exponent can be utilized as a mathematical tool to understand properties of certain operators which explains the behavior of certain systems. The readers that are not familiar with this subject may directly jump to Example 4.2 keeping in mind that Lyapunov exponents for random matrices with determinant one, i.e., over the group $\mathrm{SL}(d,\mathbb{R})$, have many applications in physics. For the more experienced reader we would like to give some further context why we consider Example 4.2 below.

Consider a random Schrödinger operator

[TABLE]

where $\Delta$ is the Laplacian and $V_{\omega}$ is a random potential. One goal is to identify typical spectral properties of such operators. In the one-dimensional case it is known that the operator $H_{\omega}$ has a complete set of eigenvectors that decay exponentially in space (see e.g. [1]). To formally prove this statement the maximal Lyapunov exponent is useful. For an energy $E\in\mathbb{R}_{+}$ we can write the one-dimensional discrete Schrödinger equation as a difference equation of the form

[TABLE]

The maximal Lyapunov exponent for a given energy $E$, denoted by $\gamma_{1}(E)$ describes the exponential growth or decay of the solution to (90). By iteration we find

[TABLE]

The sequence of random matrices $(T_{i})_{i\in\mathbb{N}}$ defined above are called transfer matrices and the corresponding maximal Lyapunov exponent happens to be inverse proportional to the localization length. In the traditional study of random Schrödinger operators we use a lower bound on $\gamma_{1}$ (e.g., via the Furstenberg theorem [23]) to ensure that there exists a finite localization length. On the other hand, an upper bound on $\gamma_{1}$ gives a lower bound on the localization length. In other words an upper bound on $\gamma_{1}$ gives an ultimate limit how small the localization length can be at most which is of general interest. There is a rich literature about properties of random operators. The interested reader can find more information about this subject in [10, 1].

Example 4.2**.**

Let $(L_{i})_{i\in\mathbb{N}}$ be i.i.d. random matrices on $\mathrm{SL}(2,\mathbb{R})$ of the form

[TABLE]

where $(\omega_{i})_{i\in\mathbb{N}}$ are i.i.d. random variables uniformly distributed over $\{-1,1\}$. A well-known result by Furstenberg [23] implies that $\gamma_{1}>0$. Using the $\mathrm{SL}(2,\mathbb{R})$ group structure we find

[TABLE]

and hence Corollary 3.3 gives

[TABLE]

Analogously we find

[TABLE]

where the second step uses that $c=\frac{4(a^{2}+1)}{a}$ is the minimizer. We can use the structure of $\mathrm{SL}(2,\mathbb{R})$ to further improve the upper bound for $\gamma_{1}$. Note that $A\in\mathrm{SL}(2,\mathbb{R})$ implies $\sigma_{1}(A)=\frac{1}{\sigma_{2}(A)}$. Hence, using (4) we find $\mathbb{P}-$a.s.

[TABLE]

where we used (96) in the final step. Thus together with Furstenberg [23] we find $\gamma_{2}\geq-0.06$ and $\gamma_{1}\in(0,0.06]$ which is a very accurate localization of the true value of $\gamma_{1}$. As a comparison, the simple bounds from Remark 3.7 give $\gamma_{1}\leq\frac{1}{2}\log\frac{1}{2}(3+\sqrt{5})\approx 0.48$ and $\gamma_{2}\geq\frac{1}{2}\log\frac{1}{2}(3-\sqrt{5})\approx-0.48$.

4.3 Other interesting examples

In this section we discuss various other examples that illustrate how to use the bounds from Corollaries 3.3 and 3.4 in practice.

Example 4.3** (Rank-one matrices).**

Let $(L_{i})_{i\in\mathbb{N}}$ be i.i.d. matrices chosen uniformly over

[TABLE]

For this scenario we have $\gamma_{1}=-\frac{1}{4}\log 2$ because each time in the large matrix product the matrix changes, which happens with probability $\frac{1}{2}$, we pick up a factor $2^{-1/2}$. Hence in the operator norm of the definition of the Lyapunov exponent we get a factor $2^{-n/4}$. To apply Corollary 3.3 we first note that the distribution for the random matrices in this example is over a semigroup

[TABLE]

and hence

[TABLE]

Corollary 3.3 then gives $\gamma_{1}\leq-\frac{1}{4}\log 2\approx-0.1733$ which is tight for this example. The simple bound from Remark 3.7 gives $\gamma_{1}\leq 0$. The known bound from [40, Equation (23)] gives $\gamma_{1}\leq-0.0792$.

Example 4.4** (Group structure).**

Let $(L_{i})_{i\in\mathbb{N}}$ be i.i.d. matrices chosen uniformly over

[TABLE]

For this scenario it has been shown that $\gamma_{1}=0$ [7, Section II.6]. Before applying the bounds from Corollary 3.3 we note that the distribution for the random matrices in this example is over a group

[TABLE]

and hence

[TABLE]

Hence Corollary 3.3 gives

[TABLE]

where the maximizer is $c^{*}=\sqrt{\frac{19}{29}}$. Analogously we also find $\gamma_{2}\geq\frac{1}{4}\log\frac{3}{4}\approx-0.07$. The simple bounds from Remark 3.7 give $\gamma_{1}\leq\frac{1}{4}\log 6\approx 0.45$ and $\gamma_{2}\geq-\frac{1}{4}\log 6\approx-0.45$.

Example 4.5** (Corollary 3.4 may outperform Corollary 3.3).**

Let $(L_{i})_{i\in\mathbb{N}}$ be i.i.d. random matrices on $\mathbb{C}^{2\times 2}$ of the form

[TABLE]

for $\alpha,\beta>0$, $\alpha\neq\beta$, and $(U_{i})_{i\in\mathbb{N}}$ randomly chosen according to the Haar measure on the unitary group $\mathrm{U}(2,\mathbb{C})$. Corollary 3.3 (which in this case coincides with the bound from [40, Equation (23)]) gives

[TABLE]

where the second step uses that by symmetry the maximizer is $\frac{1}{2}\mathrm{id}_{2}$. Corollary 3.4 gives on the other hand gives

[TABLE]

where we also used the symmetry of the Haar measure. The lower bound for $\gamma_{2}$ from Corollary 3.4 ensures that (107) is actually tight since

[TABLE]

where we again used the symmetry of the Haar measure. Hence we can conclude that

[TABLE]

Jensen’s inequality (which is strict as $\alpha\neq\beta$) assures that (107) is strictly better than (106) since

[TABLE]

where the penultimate step uses Hölder’s inequality [6, Exercise IV.2.7]. The final step is true because $\mathbb{E}\,U(1,0)^{\dagger}(1,0)U^{\dagger}=\mathrm{id}_{2}/2$. We note that depending on the value of $\alpha$ and $\beta$ the difference between (106) and (107) can be substantial. As an example for $\alpha=5$ and $\beta=1$ we obtain

[TABLE]

The simple bounds from Remark 3.7 give $\gamma_{1}\leq\log\max\{\alpha,\beta\}\approx 1.61$ and $\gamma_{2}\geq\log\min\{\alpha,\beta\}=0$.

Example 4.6** (Convex optimization solver to compute the bound).**

This example shows that in case of random matrices without any useful structure it is relevant that the upper bound from Proposition 3.6 can be evaluated efficiently using convex programming. Let $(L_{i})_{i\in\mathbb{N}}$ be i.i.d. matrices chosen uniformly over

[TABLE]

Proposition 3.6 gives

[TABLE]

where we used a convex optimization solver to compute the maximization. Duality theory of convex programming ensures the result is correct up to an error of order $O(10^{-9})$.101010We used CVX on Matlab to solve the convex optimization problem. On a MacBook with 2.3 GHz Intel Core i7 and 8 GB memory we can run examples up to $d=500$ in a reasonable time (i.e., in a few minutes). As a comparison, the simple bound from Remark 3.7 gives $\gamma_{1}\leq 3.05$.

5 Connections between Lyapunov exponents and entropy

In this section we discuss two connections between Lyapunov exponents and entropy. In particular we show that our bounds on $\gamma_{1}$ and $\gamma_{d}$ imply new relations for entropic quantities.

5.1 Towards an entropy accumulation theorem for relative entropies

Before we could prove the bounds on the maximal and minimal Lyapunov exponent we had to gain insight into what form they could have. A recent result from quantum information theory called entropy accumulation theorem [16, 15] turned out to provide evidence on the structure of the bounds. On an informal level entropy accumulation ensures that the operationally relevant quantities of a multiparty system (called smooth min-and max-entropies [31]) can be bounded by the sum of the von Neumann entropies of its individual parts viewed on a worst case scenario. Finally in the process of simplifying our argument we discovered the elementary proof for the main result that is presented in this manuscript.

To make this connection more precise we show how a variant of the entropy accumulation theorem can be obtained as direct consequence of Theorem 3.1. More precisely, we show that

[TABLE]

where $(\mathcal{M}_{i})_{i\in\mathbb{N}}$ are completely positive maps from $R\to A_{i}\otimes R$ defined by

[TABLE]

Inequality (115) may be viewed as a variant of the entropy accumulation theorem as it ensures that the relative entropy of a large system, represented by the left-hand side of (115) can be bounded as a sum of relative entropies of the individual systems.

In the following we show how (115) follows as a direct consequence from Theorem 3.1. Consider a sequence $(L_{i})_{i\in\mathbb{N}}$ of independent random matrices on $\mathbb{C}^{d}\simeq R$ with joint distribution $\mu_{L^{n}}$. By definition of the expectation value we have

[TABLE]

Furthermore, by definition of the maps $(\mathcal{M}_{i})_{i\in\mathbb{N}}$ given in (116) we find that gives

[TABLE]

Recalling that the quantum relative entropy is given by $D(\rho\|\sigma):=\mathrm{tr}\,\rho\log\rho-\mathrm{tr}\,\rho\log\sigma$ leads to

[TABLE]

Furthermore, we find for all $i\in\mathbb{N}$

[TABLE]

Theorem 3.1 in combination with (120) and (123) thus implies (115).

This connection raises further questions such as the existence of a more general entropy accumulation result for relative entropies (where the first argument is not necessarily the identity operator) that contains the bounds on the Lyapunov exponents and the original entropy accumulation theorem for conditional entropies as a special case. Another step towards a fully general entropy accumulation theorem for relative entropies has been obtained recently in [19] where a novel chain rule for the relative entropy is proven.

5.2 Entropy rate of hidden Markov processes

In this section it is shown that the entropy rate of hidden Markov processes is directly related to the maximal Lyapunov exponent. Hence the results above can be used to obtain upper and lower bounds for the entropy rate.

Let $(X_{i})_{i\in\mathbb{N}}$ be a stochastic stationary process. The entropy rate of this process is defined as

[TABLE]

where the limit exists as the process is stationary. The interested reader may consult [13] for more information about this quantity. The celebrated Shannon-McMillan-Breiman (see, e.g. [2]) theorem asserts that if $(X_{n})_{\in\mathbb{N}}$ is also ergodic we have

[TABLE]

Let $(X_{n})_{n\in\mathbb{N}}$ be a stationary and ergodic Markov process taking values in a finite set $\mathcal{X}$ described by a transition matrix $M\in[0,1]^{|\mathcal{X}|\times|\mathcal{X}|}$ such that $M_{x,x^{\prime}}=\mathbb{P}(X_{i+1}=x^{\prime}|X_{i}=x)$. Let $(Y_{n})_{n\in\mathbb{N}}$ denote a noisy version of the Markov process where the noise is described by a discrete memoryless channel.111111A channel is said to be memoryless if the probability distribution of the output depends only on the input at that time and is conditionally independent of previous channel inputs or outputs. A discrete channel consists of a discrete input alphabet $\mathcal{X}$, a discrete output alphabet $\mathcal{Y}$, and a probability transition matrix $W\in[0,1]^{|\mathcal{X}|\times|\mathcal{Y}|}$ such that $W_{x,y}=\mathbb{P}(Y_{i}=y|X_{i}=x)$.

The process $(Y_{n})_{n\in\mathbb{N}}$ is a hidden Markov process. These processes are well-studied and arise naturally in many areas of science ranging from statistics via communication and information theory to machine learning, just to name a few. The interested reader can find an extensive discussion about hidden Markov processes and their applications in [18] and references therein.

Computing the entropy rate of a hidden Markov process is a complicated task and in general an explicit form is unknown. Interestingly the entropy rate of a hidden Markov process is closely related to the maximal Lyapunov exponent. A standard recursion [18, 37] yields

[TABLE]

where $\mu$ is the stationary distribution of the Markov process $(X_{n})_{n\in\mathbb{N}}$ (represented as a column vector), $M\odot W_{\cdot,y_{i}}^{\mathrm{T}}$ denotes the denotes the $|\mathcal{X}|\times|\mathcal{X}|$ matrix whose $x$th row is given by the componentwise multiplication of the $x$th row of $M$ by the row vector whose $x^{\prime}$th component is $W_{x^{\prime},y_{i}}$, and $\mathds{1}$ is the all-1 column vector. Since for a matrix $A$ with nonnegative entries $\mu^{\mathrm{T}}A\mathds{1}$ is a norm of $A$ and since all matrix norms are equivalent we have

[TABLE]

In words, the entropy rate of the hidden Markov process $(Y_{n})_{n\in\mathbb{N}}$ is equal to the negative maximal Lyapunov exponent of the random matrices $(M\odot W_{\cdot,Y_{n}}^{\mathrm{T}})_{n\in\mathbb{N}}$. This connection was also observed and discussed in [26, 27].

Acknowledgments.

We thank Jürg Fröhlich and Marius Lemm for discussions on random matrices and localization of random operators. We also thank Gian Michele Graf for making us aware of related literature, in particular [40, Equation (23)]. We further thank Lennart Baumgärtner, Raban Iten, and Tobias Sutter for inspiring discussions about Lyapunov exponents [4] and entropy rates of hidden Markov processes. This work was funded by the Swiss National Science Foundation via project No. 200020_165843 and via the National Centre of Competence in Research QSIT, by the Air Force Office of Scientific Research (AFOSR) via grant FA9550-19-1-0202, as well as by the French ANR project ANR-18-CE47-0011 (ACOM).

Appendix

Appendix A Lyapunov spectrum exists for stationary random matrices

In this section we prove that for any sequence $(L_{n})_{n\in\mathbb{N}}$ of stationary random matrices on $\mathbb{C}^{d\times d}$ such that $\mathbb{E}\log\sigma_{\max}(L_{1})<\infty$ the limit in

[TABLE]

exists for $1\leq k\leq d$. We note that this fact is known and is explained here for completeness.

To prove this we need to introduce the antisymmetric tensor product. For $j\in\mathbb{N}$ and a Hilbert space $\mathcal{H}$ let $\mathcal{H}^{\wedge j}$ denote the antisymmetric subspace of $\mathcal{H}^{\otimes j}$. The $j$-th antisymmetric tensor power $\wedge^{j}:\mathrm{L}(\mathcal{H})\to\mathrm{L}(\mathcal{H}^{\wedge j})$ maps every matrix $L\in\mathcal{H}$ to the restriction of $L^{\otimes j}$ to the antisymmetric subspace $\mathcal{H}^{\wedge j}$ of $\mathcal{H}^{\otimes i}$, where $\mathrm{L}(\mathcal{H})$ denotes the set of matrices on $\mathcal{H}$. This mapping is well studied and oftentimes serves as a useful tool in proofs. Among other interesting properties [6, Section I.5 and p. 18] it satisfies for any $L_{1},L_{2}\in\mathbb{C}^{d\times d}$ and any $j\in\mathbb{N}$

[TABLE]

and

[TABLE]

By property (129) above and the submultiplicativity of the largest singular value [6] we have for $n,m\in\mathbb{N}$

[TABLE]

where the penultimate step uses the assumption that $(L_{n})_{n\in\mathbb{N}}$ are stationary. We thus see that $(a_{j,n})_{n\in\mathbb{N}}$ is a subadditive sequence and hence according to Fekete’s subadditivity lemma [20] the limit

[TABLE]

exists.

For $j=1$ we find with the help of (130)

[TABLE]

i.e., the asserted limit in (128) exists for $k=1$. For $j=2$ again using (130) gives

[TABLE]

which shows that the asserted limit in (128) exists for $k=2$. We can now continue this argument to show that the limit in (128) exists for all $k\in[d]$. ∎

Appendix B Proof of Theorem 2.1

Because the reference [3] is in preparation we present a proof for the assertion of Theorem 2.1 in Appendix B for completeness.

Let $C$ be a topological vector space and $C^{*}$ its continuous dual. The weak* topology on $C^{*}$ is defined to be the $C$-topology on $C^{*}$, i.e., the coarsest topology (the topology with the fewest open sets) under which every element $c\in C$ corresponds to a continuous map on $C^{*}$. We now take $C$ to be the space of continuous functions with the supremum norm on $\mathrm{GL}(d,\mathbb{C})$. By the Riesz-Markov theorem its dual space $C^{*}$ can be identified with the space of all complex regular Borel measures of bounded variation on $\mathrm{GL}(d,\mathbb{C})$ and the space of compactly supported probability measures on $\mathrm{GL}(d,\mathbb{C})$, denoted by $\bar{G}(d,\mathbb{C})$, can be identified with a subspace thereof. The weak* topology on $C^{*}$ can then be restricted to a topology on $\bar{G}(d,\mathbb{C})$. Let $\bar{G}(d,\mathbb{C})$ be equipped with the weakest topology $\mathcal{T}$ such that

$\mathcal{T}$ is stronger than the weak* topology restricted to $\bar{G}(d,\mathbb{C})$ 2. 2.

$\mathcal{T}$ is stronger than the pull-back of the Hausdorff topology by $\mu\mapsto\mathrm{supp}\,\mu$.

More information about these assumptions can be found in [44, 21].

It has been shown [44, Theorem 3.5] that $\mu\mapsto\gamma_{1}(\mu)$ is continuous on $\bar{G}(d,\mathbb{C})$, where $\gamma_{1}$ denotes the maximal Lyapunov exponent.121212We note that for $2\times 2$ matrices the continuity of the maximal Lyapunov exponent has been proven in [45]. This implies that for probability measures supported on finite sets, i.e.,

[TABLE]

the maximal Lyapunov exponent varies continuously with the probabilities $p_{i}>0$ and the matrices $L_{i}\in\mathrm{GL}(d,\mathbb{C})$ at every point. To see this it suffices to show that the function $(p,L)\mapsto\sum_{i=1}^{n}p_{i}\delta_{L_{i}}$ is continuous for $p=(p_{1},\ldots,p_{n})$ and $L=(L_{1},\ldots,L_{n})$, because $\mu\mapsto\gamma_{1}(\mu)$ is known to be continuous. Since $\mathcal{T}$ is defined as the weakest topology satisfying points 1. and 2., any open set of $\bar{G}(d,\mathbb{C})$ is a union of an intersection between an open set according to the weak* topology and an open set according to the pull-back of the Hausdorff topology under the $\mathrm{supp}$ map.131313To see why considering a union of finite intersections is sufficient, let $T^{\prime}$ and $T^{\prime\prime}$ be two topologies on the same set. The weakest topology $T$ on that set that is stronger than $T^{\prime}$ and $T^{\prime\prime}$ may be defined as the topology consisting of all open sets of the form

$\displaystyle Z=\cup_{i\in\mathcal{S}}X_{i}\cap Y_{i}\,,$

(140)

where $(X_{i})$ and $(Y_{i})$ are families of open sets of $T^{\prime}$ and $T^{\prime\prime}$, respectively, parameterized by a (not necessarily countable) set $\mathcal{S}$. Any (finite) intersection of sets $Z$ of the form (140) is again of the form (140). The same is obviously true for (not necessarily finite) unions. So the set of all sets $Z$ of the form (140) is a valid topology. Furthermore, any $X\in T^{\prime}$ and any $Y\in T^{\prime\prime}$ can also be expressed in the form (140), as one can just take one $X_{i}$ to be equal to $X$ and $Y_{i}$ equal to the entire set, or analogously for $Y$. Hence $T$ is indeed at least as strong as $T^{\prime}$ and $T^{\prime\prime}$. It is therefore sufficient to prove that the pre-image of any such set under the map $(p,L)\mapsto\sum_{i=1}^{n}p_{i}\delta_{L_{i}}$ is open. This is done in two steps.

First, let $X$ be a subset of $\bar{G}(d,\mathbb{C})$ that is open according to the weak* topology. This means that $X$ is a union of finite intersections of pre-images of open subsets of $\mathbb{R}$ under any map $\mu\mapsto\int\phi\mu$, where $\phi$ is a “test function”, i.e., a function in $C$. But this means that the pre-image of $X$ under (139) is a union of intersections of pre-images of open subsets of $\mathbb{R}$ under the concatenated map $(p,L)\mapsto\int[\phi\sum_{i}p_{i}\delta_{L_{i}}]=\sum_{i}p_{i}\phi(L_{i})$. Since $\phi$ is continuous, this concatenated map is continuous, and hence its pre-images are indeed open (for any $p_{i}\geq 0$).

Second, let $X$ be a subset of $\bar{G}(d,\mathbb{C})$ that is open according to the the pull-back of the Hausdorff topology under the supp map. By definition, this means that $X$ is itself the pre-image of an open set (on the set of subsets of $\mathrm{GL}(d,\mathbb{C})$) under the supp map. But this means that the pre-image of $X$ is the pre-image of an open set (on the set of subsets of $\mathrm{GL}(d,\mathbb{C})$) under the concatenated map $(p,L)\mapsto\textrm{supp}(\sum_{i}p_{i}\delta_{L_{j}})=\{L_{i}:i\textnormal{ such that }p_{i}>0\}$. For $p_{i}>0$ this map is continuous according to the Hausdorff topology, and hence pre-images of open sets are open.