q-deformed Character Theory for Infinite-Dimensional Symplectic and Orthogonal Groups

TL;DR

This paper extends the classification of irreducible spherical characters to q-deformed orthogonal and symplectic groups, providing new formulas and methods for understanding their asymptotic behavior.

Contribution

It introduces a q-deformed character theory for infinite-dimensional orthogonal and symplectic groups, expanding previous results from the unitary case.

Findings

Derived determinantal formulas for q-specialized characters

Established double-contour integral representations

Extended classification results to new group types

Abstract

The classification of irreducible, spherical characters of the infinite-dimensional unitary/orthogonal/symplectic groups can be obtained by finding all possible limits of normalized, irreducible characters of the corresponding finite-dimensional groups, as the rank tends to infinity. We solve a q-deformed version of the latter problem for orthogonal and symplectic groups, extending previously known results for the unitary group. The proof is based on novel determinantal and double-contour integral formulas for the q-specialized characters.

Equations

$$\underset{N\to \infty }{\lim }\frac{s_{\lambda }(x_1,x_2,\dots ,x_k,1^{N-k})}{s_{\lambda }(1^N)},k=1,2,\dots ,|x_1|=|x_2|=\cdots =|x_k|=1,$$

$$\underset{N\to \infty }{\lim }\frac{s_{\lambda (N)}(x_1,x_2,\dots ,x_k,q^{-k},q^{-k-1},\dots ,q^{1-N})}{s_{\lambda }(1,q^{-1},q^{-2},\dots ,q^{1-N})}.$$

$${\nu }_i=\underset{N\to \infty }{\lim }\lambda (N{)}_{N+1-i},i=1,2,\dots .$$

$$\underset{N\to \infty }{\lim }\frac{s_{\lambda (N)}(q^{-N},q^{1-N},\dots ,q^{-k-1},x_{-k},\dots ,x_k,q^{k+1},q^{k+1},\dots ,q^N)}{s_{\lambda (N)}(q^{-N},q^{1-N},\dots ,q^{N-1},q^N)},x_1,\dots ,x_k\in {\mathbb{C}}^{\ast },$$

$$\underset{N\to \infty }{\lim }\lambda (N{)}_i={\nu }_i\text{ for all }i\in \mathbb{Z}.$$

$$\underset{N\to \infty }{\lim }\frac{{\chi }_{\lambda (N)}^G(x_1,\dots ,x_k,q^{k+ϵ},\dots ,q^{N-1+ϵ})}{{\chi }_{\lambda (N)}^G(q^{ϵ},q^{1+ϵ},\dots ,q^{N-1+ϵ})},x_1,\dots ,x_k\in {\mathbb{C}}^{\ast },$$

$$\underset{N\to \infty }{\lim }\lambda (N{)}_{N+1-i}={\nu }_i,i=1,2,\dots .$$

$$(a;q{)}_n:=\prod _{i=1}^n(1-a{q}^{i-1}),n\ge 0;(a;q{)}_{\infty }:=\prod _{i=1}^{\infty }(1-a{q}^{i-1}).$$

$$(a_1,\dots ,a_m;q{)}_n:=(a_1;q{)}_n\cdots (a_m;q{)}_n;(a_1,\dots ,a_m;q{)}_{\infty }:=(a_1;q{)}_{\infty }\cdots (a_m;q{)}_{\infty }.$$

$$s_{\lambda }(x_1,\dots ,x_N):=\frac{{\det }_{1\le i,j\le N}\left[x_i^{{\lambda }_j+N-j}\right]}{V(x_1,\dots ,x_N)},$$

$$V(x_1,\dots ,x_N):=\prod _{1\le i<j\le N}(x_i-x_j)$$

$$s_{\lambda }(x_1,\dots ,x_N,u)=\sum _{\begin{array}{c}\mu \in {\mathbb{GT}}_N\\ \mu \prec \lambda \end{array}}{s}_{\mu }(x_1,\dots ,x_N)u^{|\lambda |-|\mu |}.$$

$$\frac{s_{\lambda }(q^{{\mu }_1+N-1},\dots ,q^{{\mu }_{N-1}+1},q^{{\mu }_N})}{s_{\lambda }(q^{N-1},\dots ,q,1)}=\frac{s_{\mu }(q^{{\lambda }_1+N-1},\dots ,q^{{\lambda }_{N-1}+1},q^{{\lambda }_N})}{s_{\mu }(q^{N-1},\dots ,q,1)}.$$

$$s_{\lambda }(1,q,\dots ,q^{N-1})=q^{n(\lambda )}\prod _{1\le i<j\le N}\frac{1-q^{{\lambda }_i-{\lambda }_j+j-i}}{1-q^{j-i}}.$$

$$s_{(a+1,1^b,0^{N-b-1})}(x_1,\dots ,x_N)=\sum _{i=0}^b(-1{)}^i{h}_{a+1+i}(x_1,\dots ,x_N)e_{b-i}(x_1,\dots ,x_N).$$

$$SO(2N+1),Sp(N),SO(2N).$$

$$Sp(2N):=\left\{A\in GL(2N,\mathbb{C}):A^{t}BA=B,B=\left[\begin{array}{cc}0& I_N\\ I_N& 0\end{array}\right]\right\}$$

$$Sp(N):=Sp(2N)\cap U(2N).$$

$$\begin{gathered} {\chi }_{\lambda }^G(z_1,\dots ,z_N):= \\ \{\begin{array}{ll}s{o}_{\lambda }^{2N+1}(M),& \text{if }G=B;\text{ M∈SO(2N+1) has eigenvalues {zi,zi−1}i=1n∪{1}};\\ s{p}_{\lambda }^N(M),& \text{if }G=C;\text{ M∈Sp(N) has eigenvalues {zi,zi−1}i=1n};\\ s{o}_{\lambda }^{2N}(M),& \text{if }G=D;\text{ M∈SO(2N) has eigenvalues {zi,zi−1}i=1n}.\end{array} \end{gathered}$$

$$V^s(z_1,\dots ,z_N):=\prod _{1\le i<j\le N}(z_i+z_i^{-1}-z_j-z_j^{-1}),$$

$${\chi }_{\lambda }^G(z_1,\dots ,z_N)=\frac{1}{V^s(z_1,\dots ,z_N)}\times \{\begin{array}{ll}\underset{1\le i,j\le N}{\det }\left[\frac{z_j^{{\lambda }_i+N-i+\frac{1}{2}}-z_j^{-{\lambda }_i-N+i-\frac{1}{2}}}{z_j^{\frac{1}{2}}-z_j^{-\frac{1}{2}}}\right],& \text{if }G=B,\\ \underset{1\le i,j\le N}{\det }\left[\frac{z_j^{{\lambda }_i+N-i+1}-z_j^{-({\lambda }_i+N-i+1)}}{z_j-z_j^{-1}}\right],& \text{if }G=C,\\ \underset{1\le i,j\le N}{\det }\left[z_j^{{\lambda }_i+N-i}+z_j^{-({\lambda }_i+N-i)}\right],& \text{if }G=D.\end{array}$$

$${\chi }_{\lambda }^G(z_1,\dots ,z_N,u)=\sum _{\mu \in {\mathbb{GT}}_N^{+}}{\chi }_{\lambda /\mu }^G(u){\chi }_{\mu }^G(z_1,\dots ,z_N),$$

$${\chi }_{\lambda /\mu }^G(u):=\sum _{\begin{array}{c}\nu \in {\mathbb{GT}}_{N+1}^{+}\\ \lambda \succ \nu \succ \mu \end{array}}{\tau }^G(u;\lambda ,\nu ,\mu )u^{2|\nu |-|\lambda |-|\mu |},$$

$${\tau }^B(u;\lambda ,\nu ,\mu )$$

$${\tau }^C(u;\lambda ,\nu ,\mu )$$

$${\tau }^D(u;\lambda ,\nu ,\mu )$$

$${\chi }_{\lambda /\mu }^G(u)=\sum _{\begin{array}{c}\nu \in {\mathbb{GT}}_{N+1}^{+}\\ \lambda \succ \nu \succ \mu \end{array}}{\tau }^G(1/u;\lambda ,\nu ,\mu )u^{|\lambda |+|\mu |-2|\nu |}.$$

$$ϵ:=\{\begin{array}{ll}1/2,& \text{if }G=B,\\ 1,& \text{if }G=C,\\ 0,& \text{if }G=D.\end{array}$$

$$l_i^G:={\lambda }_i+N-i+ϵ,n_i^G:={\nu }_i+N-i+ϵ,\text{ for all }i=1,2,\dots ,N.$$

$$\frac{{\chi }_{\lambda }^G(q^{n_1},q^{n_2},\dots ,q^{n_N})}{{\chi }_{\lambda }^G(q^{N-1+ϵ},q^{N-2+ϵ},\dots ,q^{ϵ})}=\frac{{\chi }_{\nu }^G(q^{l_1},q^{l_2},\dots ,q^{l_N})}{{\chi }_{\nu }^G(q^{N-1+ϵ},q^{N-2+ϵ},\dots ,q^{ϵ})}.$$

$${\chi }_{\lambda }^B(q^{\frac{1}{2}},q^{\frac{3}{2}},\dots ,q^{N-\frac{1}{2}})$$

Full text

1. 1 Introduction

2. 1.1 Preface

3. 1.2 $q$–deformed characters: main results

4. 1.3 Further results and outlook

5. 1.4 Methodology

6. 1.5 Terminology and conventions

7. 1.6 Acknowledgements

8. 2 Characters of $U(N)$, $SO(N)$ and $Sp(N)$

9. 2.1 Type A characters

10. 2.2 Type B-C-D characters

11. 3 Asymptotics of $q$-deformed characters

12. 3.1 Type A characters: statements of results

13. 3.2 Type A characters: proofs of results

14. 3.3 Type B-C-D characters: statements of results

15. 3.4 Integral representations and multivariate formulas for B-C-D type characters

16. 3.5 Type B-C-D characters: proofs of results

17. 4 Branching graphs

18. 4.1 Generalities on Branching Graphs

19. 4.2 Symmetric $q$-Gelfand-Tsetlin graph

20. 4.3 BC type $q$-Gelfand-Tsetlin graph

21. 5 Boundary of the symmetric $q$-Gelfand-Tsetlin graph

22. 5.1 A family of coherent probability measures

23. 5.2 Concentration bound

24. 5.3 Law of Large Numbers and the Martin boundary

25. 5.4 Characterization of the minimal boundary

26. 6 Boundary of the BC type $q$-Gelfand-Tsetlin graph

27. 6.1 A family of coherent probability measures

28. 6.2 Concentration bound

29. 6.3 Law of Large Numbers and the Martin boundary

30. 6.4 Characterization of the minimal boundary

1. Introduction

1.1. Preface

For each of the three series of classical compact Lie groups: unitary $U(N)$, orthogonal $SO(N)$, and symplectic $Sp(N)$, one can naturally embed groups of the smaller rank into the larger ones and form inductive limits $U(\infty)=\bigcup_{N=1}^{\infty}U(N)$, $SO(\infty)=\bigcup_{N=1}^{\infty}SO(N)$, $Sp(\infty)=\bigcup_{N=1}^{\infty}Sp(N)$. The study of such infinite–dimensional or “big” groups has been a central topic of the asymptotic representation theory during the last 40 years. These groups are wild, which means that one needs to restrict the class of representations, in order to get a meaningful theory. One point of view is to deal with characters, i.e. central, positive–definite, continuous functions on the group. The extreme characters then correspond to finite factor representations of the group; for $U(\infty)$ all of them were classified by Voiculescu [Vo], while for $SO(\infty)$ and $Sp(\infty)$ similar results were obtained by Boyer [Bo2]. More general characters can be identified with spherical representations of the Gelfand pairs $(U(\infty)\times U(\infty),U(\infty))$, $(SO(\infty)\times SO(\infty),SO(\infty))$, $(Sp(\infty)\times Sp(\infty),Sp(\infty))$, and such a theory was introduced and thoroughly studied by Olshanski [O1], [O2]. From another direction, as first noticed in [VK2],[Bo1], one can identify extreme characters with totally–positive Toeplitz matrices, and then their classification theorem becomes equivalent to the earlier Edrei’s theorem [E] from classical analysis.

Yet another approximative approach was suggested by Vershik and Kerov [VK2]; in this approach, the parameters of the characters of $U(\infty)$, $SO(\infty)$, $Sp(\infty)$ become limits of normalized lengths as $N\to\infty$ of rows and columns in the Young diagrams parameterizing the irreducible representations of finite–dimensional groups $U(N)$, $SO(N)$, $Sp(N)$. This idea can be used to produce several distinct proofs of the character classification theorems of Voiculescu and Boyer, see [OO2], [OO3], [BO], [Pe], [GP], [O6].

Classical Lie groups admit a $q$–deformation to quantum groups, cf. [BK], [CP], which leads to a natural question of whether a similar deformation is possible for the character theory of their infinite–dimensional versions. This question for the unitary groups $U(N)$ was first addressed by the second author [G] and based on the following observation. In the Vershik–Kerov approach, the characters of $U(\infty)$ can be treated as the limits of normalized Schur polynomials (characters of irreducible representation of $U(N)$) as the number of variables goes to infinity

[TABLE]

where $\lambda=\lambda(N)$ changes in an appropriate way (as $N\to\infty$) to guarantee the existence of the limit, and $1^{N}$ means $N$ variables all equal to $1$. Then the $q$–deformation of [G] is based on the replacement of $1^{N-k}$ and $1^{N}$ in (1.1) by the geometric series with ratio $q$. Such a point of view turned out to be fruitful: in [G] a classification theorem for the new $q$–characters was obtained (they are parameterized by nondecreasing sequences of integers) and in [BG], [C], [Pe], [GP], [GO], [O5] the topic was further developed. It was noticed in [G] that the $q$–characters have a link to the quantum traces for the representations of the quantized universal enveloping algebra $U_{q}(\mathfrak{gl}_{N})$; however, no infinite–dimensional object was constructed. A more elaborate representation–theoretic interpretation for the $q$–characters of $U(\infty)$ was presented recently by Sato [S] in the language of compact quantum groups.

After [G] appeared, an immediate question arose: can the results be extended to other root systems, i.e. to orthogonal and symplectic groups? Despite several other approaches to $q$–characters of unitary groups appearing in the subsequent years, it remained unclear whether the existence of a “good” $q$–deformation of the character theory for the infinite–dimensional group is an artifact of the root system of type $A$, or if it exists for all classical series of Lie groups? In the present article we resolve this question by constructing a rich $q$–deformed character theory for $SO(\infty)$ and $Sp(\infty)$.

1.2. $q$–deformed characters: main results

The Vershik–Kerov approach [VK2], [OO1] to the asymptotic representation theory makes the classification of all extreme characters of $U(\infty)$ equivalent to the following problem. Find all sequences of signatures (i.e. highest weights of irreducible representations) $\lambda(N)=(\lambda(N)_{1}\geq\lambda(N)_{2}\geq\dots\geq\lambda(N)_{N})\in\mathbb{Z}^{N}$, $N=1,2,\dots$, such that for each $k=1,2,\dots$, the limit (1.1) exists uniformly on the torus $\{(x_{1},\dots,x_{k})\in\mathbb{C}^{k}\,:\,|x_{i}|=1,\,1\leq i\leq k\}$. The limit itself is then identified with the value of the character on a unitary matrix from the group $U(\infty)$ with non-trivial (i.e. different from $1$) eigenvalues $x_{1},\dots,x_{k}$.

The $q$–deformation of [G] suggests to fix a parameter $0<q<1$ and find the sequences of signatures $\lambda(N)$, such that for each $k=1,2,\dots$ there exists a limit

[TABLE]

A priori in (1.2) the convergence is assumed to be uniform over $|x_{i}|=q^{1-i}$, $1\leq i\leq k$, i.e. on the torus of $q$–growing radius. However, a posteriori, (1.2) converges for all $x_{i}\neq 0$. The limit of (1.2) is then the desired $q$–character. There are several reformulations of this asymptotic problem: one of them is the identification of the minimal boundary (=extreme Gibbs measure on the space of paths) of a certain branching graph, called the $q$–Gelfand–Tsetlin graph. The vertices of this graph are labels of irreducible representations of $U(N)$, $N=1,2,\dots$, i.e. signatures, and the edges encode the branching rules upon restrictions from $U(N)$ onto $U(N-1)$. Each edge comes with a $q$–dependent weight, and the whole combinatorics can be linked to the notion of the quantum dimension for the representations of the quantized unversal envelopping algebra $U_{q}(\mathfrak{gl}_{N})$. We refer to [G] and Section 4 for the details. Another reformulation deals with $q$–Toeplitz matrices, see [G, Section 1.5].

The limits in (1.2) turn out to be parameterized by infinite sequences of integers $\nu_{1}\leq\nu_{2}\leq\nu_{3}\leq\dots$, $\nu_{i}\in\mathbb{Z}$, which are identified with the last rows of $\lambda(N)$:

[TABLE]

One clear feature of (1.2) is its asymmetry under the change $q\mapsto q^{-1}$. The properties of Schur polynomials imply that such change is equivalent to the transformation $\lambda(N)\mapsto\widetilde{\lambda}(N):=(-\lambda(N)_{N}\geq\dots\geq-\lambda(N)_{1})$ and therefore, when $q>1$, as $N\to\infty$ we would need to consider the first rows of $\lambda(N)$ instead of the last rows in (1.3).

If we now switch to orthogonal and symplectic groups, then the symmetry becomes important. Indeed, the eigenvalues for the matrices of these groups come in pairs $\{z_{i},z_{i}^{-1}\}$ and the characters are invariant under inversion of the variables. Thus, the first step towards the $q$–deformation for $SO(\infty)$, $Sp(\infty)$ is to make the setting of (1.2) symmetric by changing it to

[TABLE]

where $\lambda(N)$ now has $2N+1$ rows: $\lambda(N)_{-N}\geq\lambda(N)_{1-N}\geq\dots\geq\lambda(N)_{N}$. Note that, since the Schur polynomials are homogeneous, there is some further freedom, as we can multiply all its variables by arbitrary $q^{M}$; the most general setup is described in Section 3.1.

Our first main result (Theorem 3.3) proves that the limits of (1.4) are parameterized by two–sided sequences of integers $\dots\geq\nu_{-1}\geq\nu_{0}\geq\nu_{1}\geq\dots$ and the limit exists if and only if

[TABLE]

The limiting functions (“symmetric $q$–characters”) are given by explicit contour integral formulas.

Proceeding to the orthogonal and symplectic cases, let $G(N)$ denote either $SO(2N+1)$, or $Sp(N)$, or $SO(2N)$, corresponding to the root systems $B_{N}$, $C_{N}$, $D_{N}$, respectively. The irreducible representations are still parameterized by signatures $\lambda_{1}\geq\lambda_{2}\geq\dots\geq\lambda_{N}$, but this time all the coordinates are required to be positive.111For $SO(2N)$, $\lambda_{N}$ is allowed to be negative, but one should have $\lambda_{1}\geq\dots\geq\lambda_{N-1}\geq|\lambda_{N}|$. In this case we deal instead with the direct sum of two twin representations that differ by a flip of the sign of $\lambda_{N}$. The characters $\chi^{G}_{\lambda}$ of the representations can be then identified with symmetric Laurent polynomials in $N$ variables $z_{1},\dots,z_{N}$, invariant under the inversions $z_{i}\mapsto z_{i}^{-1}$, see Section 2.2 for more details. Set $\epsilon=1/2,1,0$ for types $B,C,D$, respectively, and consider the limits

[TABLE]

where $\lambda(N)=(\lambda(N)_{1}\geq\lambda(N)_{2}\geq\dots\geq\lambda(N)_{N}\geq 0)$. Our second main result (Theorem 3.13) proves that the limits in (1.6) are parameterized by growing sequences of nonnegative integers $0\leq\nu_{1}\leq\nu_{2}\leq\dots$ and the limit exists if and only if

[TABLE]

The limit functions are given by explicit contour integral formulas.

One might be wondering about our choice of normalization in (1.6). It has several explanations. First, the numbers $\epsilon,\epsilon+1,\dots,N-1+\epsilon$ are precisely the coordinates of $\rho$ — the half–sum of the positive roots in the corresponding root system. The value of ${\chi_{\lambda(N)}^{G}(q^{\epsilon},q^{1+\epsilon},\dots,q^{N-1+\epsilon})}$ is typically called the quantum dimension in the quantum groups literature — this matches the normalization in (1.1) by the ordinary dimension $\mathrm{Dim}(\lambda)=s_{\lambda}(1^{N})$ of the representation. From a more technical point of view, under the normalization of (1.6), the characters possess the label–variable duality, see Lemma 2.8; this duality is one of the key ingredients of our proofs.

Another point worth emphasizing is that we get two different $q$–deformed character theories for $SO(\infty)$: one from the approximation by odd-dimensional groups and another one from even–dimensional groups. In the $q=1$ case there was no difference, however the answers (e.g. the functional form of the limit in (1.6)) look different in our $q$–deformation. From the representation–theoretic point of view this can be linked to the fact that quantum groups related to $SO(2N)$ and $SO(2N+1)$ are also very different, and no embeddings of one into another are known. In the infinite–dimensional setting related to quantum groups, the distinction between $B$ and $D$ series also appeared before, e.g. in [O3].

1.3. Further results and outlook

There is another reformulation of the classification theorem for characters of $U(\infty)$, $SO(\infty)$, $Sp(\infty)$ in the language of combinatorial probability. Then one needs to describe all sequences of measures $P_{N}$, $N=1,2,\dots$ on the labels of irreducible representations of groups of rank $N$, which would agree through certain coherency relations, obtained from the branching rules for the representations. Such a reformulation is also possible in our $q$–deformed setting and we describe it in Section 4. In particular, the stabilization property of (1.5), (1.7) then turns into the Law of Large Numbers as explained in Sections 5.3 and 6.3.

We believe that there should be a way to transform our constructions from the language of special functions theory (as study of limits in (1.4), (1.6)) and from the language of combinatorial probability (as classifying the coherence family of probability measures in Section 4) into a truly representation–theoretic framework dealing with certain infinite–dimensional analogues of the quantum groups. Development of such a framework is an important open problem.

In [G] it was found that by using certain inhomogeneous analogues of Schur functions, one can encode the limits to (1.2) through simple multiplicative formulas (in particular, avoiding any contour integrals). It would be interesting to try to find similar formulas also for the symmetric setup (1.4) as well as for the $B,C,D$ series of (1.6), since the conceptual understanding for the limiting functions in (1.4) and (1.6) (given by the contour integral formulas in Theorems 3.3 and 3.13) is currently missing.

1.4. Methodology

Let us indicate one important idea that made the developments of this article possible. In [GP] an approach to the study of asymptotics of (1.2) was suggested based on contour integral formulas for $k=1$ and $k\times k$ determinantal formulas reducing the general $k$ case to the base case $k=1$. Through label–variable symmetry duality for the normalized Schur functions and analytic continuation, these formulas are dual to the closed generating function expression for complete homogeneous symmetric functions $h_{k}$ and Jacobi–Trudi formulas expressing Schur functions through $h_{k}$.

Although certain formulas for symplectic and orthogonal characters were also presented in [GP], they were not suitable for the purpose of the constructions of asymptotic representation theory (the normalization in an analogue of (1.6) was different and, more importantly, the particular choice of the geometric progression was $k$–dependent). The approach of [GP] also did not work for the symmetric case of (1.4).

In the present paper (as well as in the companion article [GS] where similar ideas are used for the study of products of random matrices) we make the following observation. Through the label–variable duality and analytic continuation, the $k=1$ versions of the expressions (1.4), (1.6) are linked to the characters corresponding to hook signatures. The latter also have explicit generating functions, though more complicated than the ones for $h_{k}$, and therefore, we need to use double contour integrals. The next step is to reduce general $k$ case to $k=1$. Instead of Jacobi–Trudi, our formulas are now related to Giambelli and Frobenius formulas, expressing characters with arbitrary signatures as determinants of hooks. We refer to Sections 3.2, 3.4 for the details.

1.5. Terminology and conventions

For convenience of the reader, we collect here some terminology that is used throughout the paper.

Unless otherwise stated, we assume $q\in(0,1)$ is a fixed parameter.

We often use the $q$-Pochhammer symbols

[TABLE]

Moreover,

[TABLE]

We also use the following terminology and conventions:

• •

We denote $\mathbb{N}:=\{1,2,3,\ldots\}$ and $\mathbb{N}_{0}:=\mathbb{N}\cup\{0\}=\{0,1,2,\ldots\}$.

• •

For $N\in\mathbb{N}$, denote by $\mathbb{GT}_{N}$ the set of signatures of length $N$, i.e., the set of $N$-tuples $\lambda=(\lambda_{1},\ldots,\lambda_{N})\in\mathbb{Z}^{N}$ such that $\lambda_{1}\geq\ldots\geq\lambda_{N}$.

• •

For $N\in\mathbb{N}$, denote by $\mathbb{GT}^{+}_{N}$ the set of nonnegative signatures of length $N$, i.e., the set of $N$-tuples $\lambda=(\lambda_{1},\ldots,\lambda_{N})\in\mathbb{GT}_{N}$ such that $\lambda_{N}\geq 0$.

• •

For $\lambda\in\mathbb{GT}_{N}$, denote $|\lambda|:=\sum_{i=1}^{N}{\lambda_{i}}$ and $n(\lambda):=\lambda_{2}+2\lambda_{3}+\ldots+(N-1)\lambda_{N}$. For $\lambda\in\mathbb{GT}_{N},\mu\in\mathbb{GT}_{N-1}$, write $\lambda\succ\mu$ (or $\mu\prec\lambda$) if $\lambda_{1}\geq\mu_{1}\geq\lambda_{2}\geq\cdots\geq\mu_{N-1}\geq\lambda_{N}$. Similarly, when $\lambda,\mu\in\mathbb{GT}^{+}_{N}$, write $\lambda\succ\mu$ if $\lambda_{1}\geq\mu_{1}\geq\lambda_{2}\geq\mu_{2}\geq\dots\geq\lambda_{N}\geq\mu_{N}$.

• •

We use $i$ as an index very often, thus we use the bold letter $\mathbf{i}:=\sqrt{-1}$ for the imaginary unit.

• •

The notation $(a^{n})$, $n\in\mathbb{N}_{0}$, indicates the string $(a,\ldots,a)$ ($n$ entries). This is an empty string if $n=0$.

1.6. Acknowledgements

We would like to thank G. Olshanski for encouraging us to study whether the extension of [G] to orthogonal and symplectic groups is possible and for a number of fruitful discussions. V.G. was partially supported by the NSF grant DMS-1664619, by the NEC Corporation Fund for Research in Computers and Communications, and by the Sloan Research Fellowship. The authors also thank the organizers of the Park City Mathematics Institute research program on Random Matrix Theory, where part of this work was carried out.

2. Characters of $U(N)$, $SO(N)$ and $Sp(N)$

2.1. Type A characters

For any $\lambda\in\mathbb{GT}_{N}$, the Schur polynomial $s_{\lambda}(x_{1},\ldots,x_{N})$ is

[TABLE]

where

[TABLE]

is the Vandermonde determinant.

Recall that the unitary group is $U(N):=\{A\in GL(N,\mathbb{C}):AA^{*}=A^{*}A=I\}$. By Weyl’s formula, $s_{\lambda}(x_{1},\ldots,x_{N})$ is the value of the character of the irreducible representation of $U(N)$ with highest weight $\lambda$ on a matrix with eigenvalues $x_{1},\ldots,x_{N}$, [FH, Ch. 24]. In general, $s_{\lambda}(x_{1},\ldots,x_{N})$ is a symmetric homogeneous Laurent polynomial of degree $|\lambda|$, and it is a polynomial when $\lambda\in\mathbb{GT}^{+}_{N}$.

Two special Schur polynomials occur for the partitions $\lambda=(m,0^{N-1})=(m,0,\ldots,0)$, for some $m\geq 0$, and $\lambda=(1^{m},0^{N-m})=(1,\ldots,1,0,\ldots,0)$, for some $N\geq m\geq 0$. In fact, $s_{(m,0^{N-1})}(x_{1},\ldots,x_{N})=h_{m}(x_{1},\ldots,x_{N})$ is the $m$-th complete homogeneous symmetric polynomial, whereas $s_{(1^{m},0^{N-m})}(x_{1},\ldots,x_{N})=e_{m}(x_{1},\ldots,x_{N})$ is the $m$-th elementary symmetric polynomial.

The reader can refer to [M, Ch. I] for a very thorough combinatorial study of Schur polynomials. Below we only list the properties that will be used in this article.

Proposition 2.1** (branching rule; [M], Ch. I, 5.11).**

For any $N\in\mathbb{N}$, $\lambda\in\mathbb{GT}_{N+1}$,

[TABLE]

Lemma 2.2** (label-variable duality).**

For any $\lambda,\mu\in\mathbb{GT}_{N}$, we have

[TABLE]

Proof.

Obvious from the definition of Schur polynomials. ∎

Proposition 2.3** ($q$-geometric specialization; [M], Ch. I, Ex. 1).**

For any $\lambda\in\mathbb{GT}_{N}$,

[TABLE]

Proposition 2.4** ([M], Ch. I, 3.9).**

For any integers $a,b\geq 0$, $N\geq b+1$, we have

[TABLE]

2.2. Type B-C-D characters

For $N\in\mathbb{N}$, let $G(N)$ be one of the rank $N$ classical compact Lie groups

[TABLE]

In these cases, $G(N)$ is of type $B,C$ or $D$, respectively.

We recall that the orthogonal group is realized as a matrix group by $O(N^{\prime}):=\{A\in GL(N^{\prime},\mathbb{R}):AA^{t}=A^{t}A=I\}$, and the special orthogonal group $SO(N^{\prime})$ is the subgroup of $O(N^{\prime})$ consisting of those matrices with determinant $1$. Similarly, the symplectic group is

[TABLE]

and the compact symplectic group (which is of our interest) is

[TABLE]

The irreducible representations of $G(N)$ are parametrized by $\mathbb{GT}^{+}_{N}$ if $G(N)$ is of type $B$ or $C$. We denote the character that corresponds to $\lambda\in\mathbb{GT}^{+}_{N}$ by $so^{2N+1}_{\lambda}$ and $sp^{N}_{\lambda}$. On the other hand, if $G(N)$ is of type $D$, then the irreducible representations of $G(N)$ are parametrized by $N$-tuples of integers $(\lambda_{1},\ldots,\lambda_{N})$ such that $\lambda_{1}\geq\ldots\geq\lambda_{N-1}\geq|\lambda_{N}|$. For any $\lambda\in\mathbb{GT}^{+}_{N}$, denote by $so_{\lambda}^{2N}$ the reducible character of $SO(2N)$ that is the sum of the twin characters corresponding to $\lambda^{+}:=(\lambda_{1},\ldots,\lambda_{N-1},\lambda_{N})$ and $\lambda^{-}:=(\lambda_{1},\ldots,\lambda_{N-1},-\lambda_{N})$.

The characters $so_{\lambda}^{2N+1},sp^{N}_{\lambda}$ and $so^{2N}_{\lambda}$ are central functions of the groups $SO(2N+1)$, $Sp(N)$ and $SO(2N)$, respectively. Consequently, if $M$ is a matrix in any of these groups, the value of a corresponding character on $M$ depends only on the eigenvalues of $M$. The eigenvalues of any matrix $M\in G(N)$ come in pairs $\{z_{i},z_{i}^{-1}\}$, $|z_{i}|=1$ (any matrix in $SO(2N+1)$ has an additional eigenvalue $1$). Therefore all the information about the characters $so_{\lambda}^{2N+1},sp^{N}_{\lambda},so^{2N}_{\lambda}$, is encoded in some functions of $N$ variables $z_{1},\ldots,z_{N}$, which are written down below in (2.1).

To uniformize notation, let $\mathbb{T}:=\{\zeta\in\mathbb{C}:|\zeta|=1\}$, and for any $\lambda\in\mathbb{GT}^{+}_{N}$, define the functions $\chi^{B}_{\lambda},\chi^{C}_{\lambda},\chi^{D}_{\lambda}:\mathbb{T}^{N}\rightarrow\mathbb{C}$ by

[TABLE]

They are symmetric with respect to permutation of their $N$ variables and also with respect to the involutions $z_{i}\mapsto z_{i}^{-1}$, i.e., they are symmetric with respect to the natural action of the Weyl group $W_{N}=(\mathbb{Z}/2\mathbb{Z})^{N}\rtimes S_{N}$. Moreover they are Laurent polynomials in $z_{1},\ldots,z_{N}$, i.e., they belong to the ring $\mathbb{C}[z_{1}^{\pm},\ldots,z_{N}^{\pm}]$. In fact, if we denote

[TABLE]

then Weyl’s formulas (e.g., see [Bu], [FH]) imply

[TABLE]

Thus we can (and we will) treat $\chi^{G}_{\lambda}(z_{1},\ldots,z_{N})$ either as $W_{N}$-symmetric Laurent polynomials or as functions on $(\mathbb{C}^{*})^{N}$, via the formulas in (2.1).

Proposition 2.5** (branching rule).**

For any $N\in\mathbb{N}$, $\lambda\in\mathbb{GT}^{+}_{N+1}$, and $G\in\{B,C,D\}$, we have

[TABLE]

where

[TABLE]

and for $\lambda\succ\nu\succ\mu$:

[TABLE]

Proof.

This proposition is a reformulation of [Pr, Prop. 10.2]. More specifically, the branching of characters $\chi_{\lambda}^{C}$ is given in part (a) of that proposition. This statement appeared first in [Zh]. The branching of characters $\chi_{\lambda}^{B}$ is given in part (b). The branching of characters $\chi_{\lambda}^{D}$ is given in part (c), which discusses the characters of certain (possibly reducible) tensor representations of $SO(2N)$. An equivalent description of the branching for the characters $\chi^{D}_{\lambda}$ appeared earlier in [KES]. ∎

Remark 2.6**.**

The Laurent polynomial $\chi_{\lambda/\mu}^{G}(u)$ is symmetric with respect to the inversion $u\leftrightarrow 1/u$, even though this is not evident from $(\ref{chilambdamu})$. Then we have the alternative formula

[TABLE]

Notation 2.7**.**

For the propositions below, and for the remainder of the paper, we consider the real parameter $\epsilon=\epsilon(G)$ defined by

[TABLE]

Also, given nonnegative signatures $\lambda,\nu\in\mathbb{GT}^{+}_{N}$, we denote

[TABLE]

These numbers are integers (when $G=C,D$) or half-integers (when $G=B$). We will simply write $l_{1},\ldots,l_{N}$ (resp. $n_{1},\ldots,n_{N}$) instead of $l_{1}^{G},\ldots l_{N}^{G}$ (resp. $n_{1}^{G},\ldots,n_{N}^{G}$), as the type $G\in\{B,C,D\}$ will be clear from the context.

Lemma 2.8** (label-variable duality).**

For $G\in\{B,C,D\}$, and any $\lambda,\nu\in\mathbb{GT}^{+}_{N}$, we have

[TABLE]

Proof.

This is a consequence of Weyl’s formulas $(\ref{eqn:weylformula})$. ∎

Proposition 2.9** ($q$-geometric specialization).**

For $G\in\{B,C,D\}$, and any $\lambda\in\mathbb{GT}^{+}_{N}$, we have

[TABLE]

Proof.

The formulas for $q$-geometric specializations can be derived from Weyl’s denominator formulas; for instance, for types C and D, see [JN, (4.4), (4.5)]. One also can give direct proofs; let us do it for type B. From $(\ref{eqn:weylformula})$, we have that $\chi_{\lambda}^{B}(q^{\frac{1}{2}},\ldots,q^{N-\frac{1}{2}})$ equals $\prod_{i=1}^{N}{(q^{\frac{1}{2}(i-\frac{1}{2})}-q^{-\frac{1}{2}(i-\frac{1}{2})})^{-1}}$ times $V^{s}(q^{\frac{1}{2}},q^{\frac{3}{2}},\ldots,q^{N-\frac{1}{2}})^{-1}$ times the determinant

[TABLE]

where we denoted $y_{i}:=q^{l_{i}}$ for all $i$. By elementary row operations, we see that the determinant above equals $\det_{1\leq i,j\leq N}\left[(y_{j}^{\frac{1}{2}}-y_{j}^{-\frac{1}{2}})^{2i-1}\right]=\prod_{i=1}^{N}{(y_{i}^{\frac{1}{2}}-y_{i}^{-\frac{1}{2}})}\times\det_{1\leq i,j\leq N}\left[(y_{j}+y_{j}^{-1}-2)^{i-1}\right]=\prod_{i=1}^{N}{(y_{i}^{\frac{1}{2}}-y_{i}^{-\frac{1}{2}})}\times\prod_{1\leq i<j\leq N}{(y_{j}+y_{j}^{-1}-y_{i}-y_{i}^{-1})}$. We get the final answer after we combine all these factors. ∎

For the next proposition, let $N$ be a fixed positive integer, and let $G\in\{B,C,D\}$. Define symmetric Laurent polynomials $\{H^{G}_{m}(x_{1},\ldots,x_{N})\}_{m\in\mathbb{Z}}$, $\{E^{G}_{m}(x_{1},\ldots,x_{N})\}_{m\in\mathbb{Z}}$ as follows:

[TABLE]

In these expressions, recall that $h_{m}$ (resp. $e_{m}$) is the $m$-th complete homogeneous (resp. elementary) symmetric polynomial.

Proposition 2.10**.**

For any integers $N,a,b$ such that $N\geq b+1$ and $a,b\geq 0$, we have

[TABLE]

Proof.

This is a well known statement, see for example [ESK, (3.14), (3.15)]. The idea for the proof is simple: for hook Young diagrams $\lambda=(a+1,1^{b})$, expand the following Jacobi-Trudi formulas (which can be found, for instance, also in [JN, Sec. 4] or [SV]) for symplectic/orthogonal polynomials along the first row:

[TABLE]

and then identify the $b\times b$ minors with the Laurent polynomials $E_{m}^{G}$ by another application of the Jacobi-Trudi formulas. ∎

3. Asymptotics of $q$-deformed characters

In this section, we study the asymptotic behavior of characters as $N\rightarrow\infty$ and all but finitely many variables are specialized in a $q$-geometric series.

3.1. Type A characters: statements of results

Definition 3.1**.**

We denote by $\mathfrak{X}$ the set of all doubly infinite, nondecreasing integer sequences, i.e.,

[TABLE]

Let $\mathbf{b}=(b(1),b(2),\ldots)\in\mathbb{N}_{0}^{\infty}$ be any sequence of nonnegative integers such that $\lim_{N\rightarrow\infty}{b(N)}=\lim_{N\rightarrow\infty}{(N-b(N))}=+\infty$. A sequence of signatures $\{\lambda(N)\in\mathbb{GT}_{N}\}_{N\geq 1}$ $\mathbf{b}$-stabilizes to $\mathbf{t}\in\mathfrak{X}$ if

[TABLE]

For $\mathbf{t}\in\mathfrak{X}$, let

[TABLE]

where the $u$-contour $\mathcal{L}$ consists of the directed lines $(+\infty-\frac{\pi\mathbf{i}}{2\ln{q}},-\infty-\frac{\pi\mathbf{i}}{2\ln{q}})$ and $(-\infty+\frac{\pi\mathbf{i}}{2\ln{q}},+\infty+\frac{\pi\mathbf{i}}{2\ln{q}})$, see Figure 1. The expressions $x^{u}$ and $x^{v}$ stand for $e^{u\ln{x}}$ and $e^{v\ln{x}}$, where $\ln{x}$ is taken over the branch of the logarithm with the cut along the negative real axis.

Lemma 3.2**.**

For any $\mathbf{t}\in\mathfrak{X}$, the function $\Phi^{\mathbf{t}}(x;q)$ defined by $(\ref{eqn:formulalimit})$ is analytic on the domain $\mathbb{C}\setminus((-\infty,0]\cup\{q^{n}:n\in\mathbb{Z}\})$. Moreover, it admits an analytic continuation to $\mathbb{C}^{*}$.

We denote the analytic continuation of $\Phi^{\mathbf{t}}(x;q)$ to $\mathbb{C}^{*}$ also by $\Phi^{\mathbf{t}}(x;q)$. For $m\in\mathbb{Z}$, let $A_{m}:\mathfrak{X}\rightarrow\mathfrak{X}$ be the map $A_{m}\mathbf{t}=\mathbf{t}^{\prime}:=(\dots\leq t_{-1}^{\prime}\leq t_{0}^{\prime}\leq t_{1}^{\prime}\leq\dots)$, $t^{\prime}_{n}:=t_{n+1-m}$. Define also the multivariate function

[TABLE]

By Lemma 3.2, the functions $\Phi^{A_{j}\mathbf{t}}(x;q)$ are analytic on $\mathbb{C}^{*}$, therefore $\Phi^{\mathbf{t}}(x_{1},\ldots,x_{k};q)$ defines an analytic function on $(\mathbb{C}^{*})^{k}$. In fact, the determinant in $(\ref{def:Phik1})$ has zeroes at the diagonals $x_{i}=x_{j}$, which cancel the poles coming from the Vandermonde determinant $V(x_{k},\ldots,x_{1})$.

Theorem 3.3**.**

Let $k\in\mathbb{N}$, and let $\mathbf{b}=(b(1),b(2),\ldots)\in\mathbb{N}_{0}^{\infty}$ be such that $\lim_{N\rightarrow\infty}{b(N)}=\lim_{N\rightarrow\infty}{(N-b(N))}=+\infty$. Also let $\{\lambda(N)\in\mathbb{GT}_{N}\}_{N\geq 1}$ be a sequence of signatures that $\mathbf{b}$-stabilizes to some $\mathbf{t}\in\mathfrak{X}$. Then

[TABLE]

holds uniformly over $(x_{1},\ldots,x_{k})$ in compact subsets of $(\mathbb{C}^{*})^{k}$. Conversely, if $\{\lambda(N)\in\mathbb{GT}_{N}\}_{N\geq 1}$ is a sequence of signatures such that the limit in the left hand side of $(\ref{mult:limit})$ exists and is uniform on compact subsets of $(\mathbb{C}^{*})^{k}$ and any $k\in\mathbb{N}$, then there exists $\mathbf{t}\in\mathfrak{X}$ such that $\{\lambda(N)\in\mathbb{GT}_{N}\}_{N\geq 1}$ $\mathbf{b}$-stabilizes to $\mathbf{t}$, and $(\ref{mult:limit})$ holds.

3.2. Type A characters: proofs of results

We need the following formulas for $q$-specializations of Schur polynomials.

Theorem 3.4**.**

Let $\lambda\in\mathbb{GT}_{N}$, $b\in\{1,2,\ldots,N-1\}$, and $a\in\mathbb{C}$; then

[TABLE]

In $(\ref{eqn:gorinsun})$, the $z$-contour encloses all poles $q^{\lambda_{i}+N-i}$, $i=1,\ldots,N$, but not the origin, while the $w$-contour encloses the origin and the $z$-contour.

Remark 3.5**.**

There are some apparent singularities in the right side of $(\ref{eqn:gorinsun})$, as a function of $a$, coming from factors in the denominators. However, the left side is a Laurent polynomial in $q^{a}$ and thus an entire function of $a$. Therefore the double integral vanishes at the apparent singularities.

Remark 3.6**.**

For the special case $b=N-1$, a similar formula was proved in [GP]; see also [C].

Proof.

The theorem was stated in [GS, (3.10)] and is similar to Theorem 3.14. We will go into the details of the proof of Theorem 3.14, so we only present a sketch of proof here.

Since both sides of (3.4) are rational functions on $q^{a}$ (for the right hand side, do a residue expansion), it suffices to prove this identity for positive integers $a\geq N$. But then we can use Lemma 2.2 to the left hand side of (3.4) and find that it is proportional (up to a factor not depending on $\lambda$) to $s_{(a+1-N,1^{N-b-1},0^{b})}(q^{\lambda_{1}+N-1},\ldots,q^{\lambda_{N}})$. Apply Proposition 2.4 to express this as

[TABLE]

Then use the following well–known generating functions for the symmetric polynomials $h_{n}$ and $e_{m}$ (see [M, Ch. I.2, (2.2) & (2.5)]):

[TABLE]

Equipped with these generating functions, write each term of (3.5) as a product of two contour integrals, whose contours are small counterclockwise oriented circles. After swapping the sum and the integration signs, we find that (3.5) equals

[TABLE]

After some simplifications, one then arrives at the double contour integral in the right hand side of (3.4). To obtain the precise factor before the integral, one also needs Proposition 2.3. ∎

Theorem 3.7**.**

Let $b\in\mathbb{N}_{0}$, $k,N\in\mathbb{N}$, be such that $b+k\leq N$, and also let $\lambda\in\mathbb{GT}_{N}$. Then

[TABLE]

Proof.

This theorem is similar to [GS, Prop. 3.1] and also to Theorem 3.18. We go into details for the proof of Theorem 3.18, but here we only give a sketch.

Since both sides are rational functions on $x_{1},\ldots,x_{k}$, it suffices to prove the identity for $x_{1}=q^{N+a_{1}-b-1},\ldots,x_{k}=q^{N+a_{k}-b-k}$, for any integers $a_{1}\geq a_{2}\geq\dots\geq a_{k}>k$. But then we can apply Lemma 2.2 to the left hand side of (3.6), and conclude that it is proportional (up to a factor independent of $\lambda$) to $s_{(a_{1},\ldots,a_{k},k^{N-b-k},0^{b})}(q^{\lambda_{1}+N-1},\ldots,q^{\lambda_{N-1}+1},q^{\lambda_{N}})$. Since the partition $(a_{1},\ldots,a_{k},k^{N-b-k})$ has Frobenius coordinates $(a_{1}-1,\dots,a_{k}-k\mid N-b-1,\ldots,N-b-k)$, we can use Giambelli formula (see [M, Ch. I.3, Example 9]) to express $s_{(a_{1},\ldots,a_{k},k^{N-b-k},0^{b})}(q^{\lambda_{1}+N-1},\ldots,q^{\lambda_{N-1}+1},q^{\lambda_{N}})$ as the determinant

[TABLE]

Multiply and divide the $(i,j)$–entry of by $s_{(a_{i}-i+1,\ 1^{N-b-j},\ 0^{b+j-1})}(1,q,\ldots,q^{N-1})$, so that we can apply Lemma 2.2 again; then the determinant (3.7) becomes

[TABLE]

Note that the $(i,j)$–entry here contains the factor $s_{\lambda}(1,q,\ldots,q^{b+j-2},q^{a_{i}+N-i},q^{b+j},\ldots,q^{N-1})$, which is precisely the Schur polynomial in the $(i,j)$–entry of the right hand side of (3.6) after specializing $x_{i}=q^{N+a_{i}-b-i}$. Finally, after some simplifications and using Proposition 2.3, we arrive at the desired (3.6). ∎

The following equivalent version of Theorem 3.4 is needed for the proof of Theorem 3.3.

Lemma 3.8**.**

Let $b,N\in\mathbb{N}$ be such that $b\in\{1,2,\ldots,N-1\}$. Let $x\in\mathbb{C}\setminus((-\infty,0]\cup\{q^{n}:n\in\mathbb{Z}\})$ be such that $q^{N-b}<|x|<q^{-(b+1)}$. Also let $\lambda\in\mathbb{GT}_{N}$ and denote

[TABLE]

Then

[TABLE]

The expressions $x^{u}$ and $x^{v}$ stand for $e^{u\ln{x}}$ and $e^{v\ln{x}}$, where $\ln{x}$ is taken over the branch of the logarithm with the cut along the negative real axis. The $u$-contour $\mathcal{L}$ consists of the directed lines $(+\infty-\frac{\pi\mathbf{i}}{2\ln{q}},-\infty-\frac{\pi\mathbf{i}}{2\ln{q}})$ and $(-\infty+\frac{\pi\mathbf{i}}{2\ln{q}},+\infty+\frac{\pi\mathbf{i}}{2\ln{q}})$, see Figure 1.

Remark 3.9**.**

The condition $q^{N-b}<|x|<q^{-(b+1)}$ is in place to assure that the double integral in $(\ref{eqn:lemmaprelimit})$ converges. In fact, as $\Re u\rightarrow\infty$, $u\in\mathcal{L}$, the integrand is of order $|xq^{b+1}|^{\Re u}$, whereas if $\Re u\rightarrow-\infty$, $u\in\mathcal{L}$, then the integrand is of order $|x^{-1}q^{N-b}|^{-\Re u}$.

Proof of Lemma 3.8.

We massage the formula from Theorem 3.4. Let $a=b+c$, and make the change of variables $z\mapsto q^{N-b}z$, $w\mapsto q^{N-b}w$. We obtain

[TABLE]

where $R$ is chosen to be any negative real number with very large absolute value. Next we make the change of variables $z=q^{u}$ and $w=q^{v}$, or equivalently, $u=\ln{z}/\ln{q}$ and $v=\ln{w}/\ln{q}$, where the principal branch of the logarithm is used. Then $(\ref{gorinsun2})$ becomes

[TABLE]

In the integral formula $(\ref{gorinsun3})$, the $u$-contour is closed, counter-clockwise oriented, encloses all points $\lambda_{i}+b-i$, $i=1,\ldots,N$, and is within the strip $\{z\in\mathbb{C}:|\Im z|<\pi\mathbf{i}/\ln{q}\}$, whereas the $v$-contour is the line $[R-\pi\mathbf{i}/\ln{q},R+\pi\mathbf{i}/\ln{q}]$ going downwards, and is to the left of the $u$-contour.

Let $x\in\mathbb{C}\setminus\left(\{0\}\cup\{q^{n}:n\in\mathbb{Z}\}\right)$ and set $c:=\ln{x}/\ln{q}$, so that $q^{c}=x$. Exchange the order of integration for the double integral in $(\ref{gorinsun3})$. Also, the factors before the integral can be written in terms of $q$-Pochhammer symbols. Then $(\ref{gorinsun3})$ is rewritten as

[TABLE]

Next we modify the contours in $(\ref{gorinsun4})$. Observe that the integrand, as a function of $u$, has poles at $u=v+2k\pi\mathbf{i}/\ln{q}$, $k\in\mathbb{Z}$, and at the points of the set

[TABLE]

and their shifts by some integral multiple of $2\pi\mathbf{i}/\ln{q}$. However, the only poles enclosed by the $u$-contour are those in $P_{N}$. Thus we can deform the $u$-contour to be the counter-clockwise oriented rectangle with vertices at the points $A\pm\frac{\pi\mathbf{i}}{2\ln{q}}$ and $B\pm\frac{\pi\mathbf{i}}{2\ln{q}}$, for any real numbers $A,B$ such that $R<A<\lambda_{N}+b-N<\lambda_{1}+b-1<B$. Call such contour $\mathcal{L}(A,B)$.

Let $\mathcal{L}$ be the infinite contour which is described in the statement of the lemma. We want to replace $\mathcal{L}(A,B)$ by $\mathcal{L}$ in the integral formula $(\ref{gorinsun4})$. For that, we need to guarantee that the absolute value of the integrand decays sufficiently fast as $|u|\rightarrow\infty$ along $\mathcal{L}$. In fact, if $\Re u$ is very large, then $\prod_{j=1}^{N-b}{(1-q^{-t_{1-j}^{N}+j+u})}\approx 1$ and the factor $x^{u}(1-q^{v-u})^{-1}\prod_{i=1}^{b}{(1-q^{t_{i}^{N}+i-1-u})^{-1}}$ decays exponentially fast as $\Re u\rightarrow\infty$ along $\mathcal{L}$, because $|xq^{b+1}|<1$. One can obtain the same conclusion if $\Re u\rightarrow-\infty$ along $\mathcal{L}$, by using instead $|x^{-1}q^{N-b}|<1$.

When we replace $\mathcal{L}(A,B)$ by $\mathcal{L}$, we pick up residues at the singularities $u=v\in[R-\frac{\mathbf{i}\pi}{2\ln{q}},R+\frac{\mathbf{i}\pi}{2\ln{q}}]$. The residue is simply $x^{v}/\ln{q}$ because all factors in the products of $(\ref{gorinsun4})$ equal $1$ when $u=v$. We conclude

[TABLE]

Finally we need to show that $(\ref{gorinsun5})$ does not depend on $R$, and thus we can set $R=0$, concluding the proof of the lemma. For that we subtract the second line of $(\ref{gorinsun5})$ with $R=R_{1}$ from the same expression with $R=R_{2}$ and show that the result is zero; without loss of generality, assume $R_{1}<R_{2}$. If we do this for the second term in the second line of $(\ref{gorinsun5})$, the residue theorem yields

[TABLE]

For the first term in the second line of $(\ref{gorinsun5})$, the residue theorem again yields

[TABLE]

where

[TABLE]

is a function of $u,v$. Observe that $F(u,v)$ depends on $v$ via the variable $q^{v}$. If $\Im v=\pm\pi\mathbf{i}/\ln{q}$, we have $q^{v}=-q^{\Re v}$, thus the integral of $F(u,v)$ with respect to $v$ along the contour $[R_{1}-\frac{\pi\mathbf{i}}{\ln{q}},R_{2}-\frac{\pi\mathbf{i}}{\ln{q}}]$ is equal to its integral along the contour $[R_{1}+\frac{\pi\mathbf{i}}{\ln{q}},R_{2}+\frac{\pi\mathbf{i}}{\ln{q}}]$; consequently the first term in the second line of $(\ref{eqn:subtraction1})$ vanishes. On the other hand, it is clear that $\textrm{Res}_{v=u}{F(u,v)}=-x^{u}/\ln{q}$ if $R_{1}<\Re u<R_{2}$ and $\textrm{Res}_{v=u}{F(u,v)}=0$ otherwise. Therefore the second line of $(\ref{eqn:subtraction1})$ equals

[TABLE]

and since $(\ref{eqn:match2})$ cancels exactly $(\ref{eqn:match1})$, we deduce the claim that $(\ref{gorinsun5})$ does not depend on $R$. ∎

We prove Theorem 3.3 and Lemma 3.2 simultaneously.

Proofs of Theorem 3.3 and Lemma 3.2.

We proceed in several steps.

Step 1. We prove the first part of Lemma 3.2, which says that the formula $(\ref{eqn:formulalimit})$ defines an analytic function on $\mathbb{C}\setminus((-\infty,0]\cup\{q^{n}:n\in\mathbb{Z}\})$. The first line of $(\ref{eqn:formulalimit})$ is clearly analytic on $\mathbb{C}\setminus\{q^{n}:n\in\mathbb{Z}\}$. As for the second line of $(\ref{eqn:formulalimit})$, the only difficulty is to argue that the double integral is an analytic function of $x$ on $\mathbb{C}\setminus(-\infty,0]$. Indeed, the factors $\prod_{i=1}^{\infty}(1-q^{t_{i}+i-1-u})^{-1}\prod_{j=1}^{\infty}(1-q^{j+u-t_{1-j}})^{-1}$ ensure that the integrand is exponentially small, as $|\Re u|\rightarrow\infty$, $u\in\mathcal{L}$, and in particular the integral converges uniformly for $x$ in compact subsets of $\mathbb{C}\setminus(-\infty,0]$. The analyticity of $\Phi^{\mathbf{t}}(x;q)$ follows.

Step 2. The general case $k>1$ of Theorem 3.3 follows from the $k=1$ case and the multivariate formula in Theorem 3.7. We are left to prove the second part of Lemma 3.2 (about the analytic continuation of $\Phi^{\mathbf{t}}(x;q)$), Theorem 3.3 for $k=1$, and the last statement of Theorem 3.3 (the converse statement).

Steps 3–4 below prove the limit $(\ref{mult:limit})$ for $k=1$, uniformly for $x$ in compact subsets of $\mathbb{C}\setminus(-\infty,0]$. Step 5 shows that $\Phi^{\mathbf{t}}(x;q)$ admits an analytic continuation to $\mathbb{C}^{*}$ and that the limit $(\ref{mult:limit})$ for $k=1$ continues to hold uniformly for $x$ in compact subsets of $\mathbb{C}^{*}$. Finally, step 6 shows the converse statement of Theorem 3.3 and finishes the proof.

Step 3. We begin with formula $(\ref{eqn:lemmaprelimit})$ from Lemma 3.8, which is valid for $x$ in the domain $\mathbb{C}\setminus((-\infty,0]\cup\{q^{n}:n\in\mathbb{Z}\})$ and $N$ large enough so that $|xq^{b(N)+1}|,|x^{-1}q^{N-b(N)}|<1$. We show that the prefactor of $(\ref{eqn:lemmaprelimit})$ converges to the prefactor of $(\ref{eqn:formulalimit})$; we also show that the integrand (of the first integral) of $(\ref{eqn:lemmaprelimit})$ converges pointwise to the integrand (of the first integral) of $(\ref{eqn:formulalimit})$. In fact, the statement about the prefactors is equivalent to the limit

[TABLE]

whereas the statement about the integrands is equivalent to the pointwise limit

[TABLE]

Both are obvious: the first comes from the assumption $\lim_{N\rightarrow\infty}{b(N)}=\lim_{N\rightarrow\infty}{(N-b(N))}=+\infty$, and the second from the limits $\lim_{N\rightarrow\infty}t_{i}^{N}=\lim_{N\rightarrow\infty}\lambda(N)_{b(N)+1-i}=t_{i}$.

Step 4. From the claim about the integrands in the previous step, we want to conclude that

[TABLE]

uniformly for $x$ on compact subsets of $\mathbb{C}\setminus(-\infty,0]$. We need the dominated convergence theorem and bounds of the prelimit integrands. The product of factors of the form $1-q^{t_{i}^{N}+i-1-v}$ and $1-q^{t_{j}^{N}+j+v}$ has a modulus that can be upper bounded by the constant $\prod_{i=1}^{\infty}(1+q^{t_{1}+i-1})(1+q^{-t_{0}+i})$, at least when $N$ is large enough so that $t_{1}^{N}=t_{1}$ and $t_{0}^{N}=t_{0}$. Because $u\in\mathcal{L}$, we have $|x^{u}|=|x|^{\Re u}e^{\pm\pi\arg(x)/2\ln{q}}\leq|x|^{\Re u}e^{-\pi^{2}/(4\ln{q})}$.

Because $q^{v-u}$ is purely imaginary for $u\in\mathcal{L}$ and $v\in[-\pi\mathbf{i}/\ln{q},\pi\mathbf{i}/\ln{q}]$, then $|1/(1-q^{v-u})|\leq 1$. Similarly, because $q^{u}$ is purely imaginary for $u\in\mathcal{L}$, the products of factors $1/(1-q^{-t_{1-j}^{N}+j+u})$ and the factors $1/(1-q^{-t_{1-j}^{N}+j+u})$ have moduli that are bounded above by $1$. Moreover, if $u\in\mathcal{L}$ and $\Re u\rightarrow+\infty$, then the product of factors $1/(1-q^{t_{i}^{N}+i-1-u})$ decays exponentially and overcomes the factor $|x|^{\Re u}$ of the previous paragraph, whereas if $u\in\mathcal{L}$ and $\Re u\rightarrow-\infty$, then the product of factors $1/(1-q^{t_{1-j}^{N}+j+u})$ is the one that decays exponentially and overcomes the factor $|x|^{\Re u}$.

All the bounds above show that the integrand in the left side of $(\ref{eqn:limitintegrals})$ converge uniformly (on $u$ and $x$) to the integrand in the right side of $(\ref{eqn:limitintegrals})$. The limit $(\ref{eqn:limitintegrals})$ follows, and the limit $(\ref{mult:limit})$ of Theorem 3.3 has been shown to hold for $k=1$ and for $x$ belonging to compact subsets of $\mathbb{C}\setminus((-\infty,0]\cup\{q^{n}:n\in\mathbb{Z}\})$.

Step 5. We show that $\Phi^{\mathbf{t}}(x;q)$ admits an analytic continuation to $\mathbb{C}^{*}$ and that $(\ref{mult:limit})$ continues to hold uniformly on compact subsets of $\mathbb{C}^{*}$. It will be convenient to use the notation

[TABLE]

Let $R>1$ be an arbitrary large positive number such that $R\notin\{q^{n}:n\in\mathbb{Z}\}$. For any $x\in\mathbb{C}^{*}$ with $R^{-1}\leq|x|\leq R$, we have $|x|^{n}<R^{n}+R^{-n}$ for any $n\in\mathbb{Z}$. By the triangle inequality and the positivity of branching coefficients of the Schur polynomials, we deduce

[TABLE]

From our choice of $R$ and steps 1–2, the limits $\lim_{N\rightarrow\infty}{\Phi_{\lambda(N)}(R^{\pm 1};q,b(N),N)}$ exist and thus, both

[TABLE]

are bounded sequences. Then the sequence of functions $\{\Phi_{\lambda(N)}(z;q,b(N),N)\}_{N\geq 1}$ is uniformly bounded on $\{z\in\mathbb{C}:R^{-1}<|z|<R\}$. Montel’s theorem implies that any subsequence of $\{\Phi_{\lambda(N)}(x;q,b(N),N)\}_{N\geq 1}$ has a subsequential limit, and the convergence is uniform on compact subsets of $\{z\in\mathbb{C}:R^{-1}<|z|<R\}$. Since $R>1$ was arbitrarily large, each subsequence of $\{\Phi_{\lambda(N)}(x;q,b(N),N)\}_{N\geq 1}$ has subsequential limits, uniformly on compact subsets of $\mathbb{C}^{*}$; say one of the limiting functions is the analytic function $\widetilde{\Phi}(x;q)$ on $\mathbb{C}^{*}$. But we already proved $(\ref{eqn:formulalimit})$ in a restricted domain so that $\widetilde{\Phi}(x;q)=\Phi(x;q)$, for $x\in\mathbb{C}\setminus((-\infty,0]\cup\{q^{n}:n\in\mathbb{Z}\})$ and such analytic continuation $\widetilde{\Phi}(x;q)$ is unique.

Step 6. Assume that $\{\lambda(N)\in\mathbb{GT}_{N}\}$ is a sequence of signatures such that the limit in the left hand side of $(\ref{mult:limit})$ exists for all $k\in\mathbb{N}$. The prelimit expression is a symmetric Laurent polynomial in $x_{1},\ldots,x_{k}$, therefore it can be written as a linear combination of Schur polynomials $s_{\mu}(x_{1},\ldots,x_{k})$, $\mu\in\mathbb{GT}_{k}$. Write the expansion as

[TABLE]

Theorem 2.1 shows that the coefficients $\Lambda^{N}_{k}(\lambda(N),\mu)$ of the expansion are nonnegative. Moreover, by plugging $x_{1}=1,x_{2}=q,\dots,x_{k}=q^{k-1}$ into $(\ref{eqn:expansionA})$, we find that $\sum_{\mu\in\mathbb{GT}_{k}}\Lambda^{N}_{k}(\lambda(N),\mu)=1$, i.e., $\Lambda^{N}_{k}(\lambda(N),\cdot)$ is a probability measure on $\mathbb{GT}_{k}$. The proof of Proposition 5.2 below shows that our assumption (existence of the limit in the left hand side of $(\ref{mult:limit})$ on compact subsets of $(\mathbb{C}^{*})^{k}$) implies the weak convergence of the sequence $\Lambda^{N}_{k}(\lambda(N),\cdot)$, as $N$ goes to infinity, to a probability measure $M_{k}$ on $\mathbb{GT}_{k}$. The proof of Theorem 5.6 below then shows that the limits

[TABLE]

exist for all $m\in\mathbb{Z}$. If we denote by $t_{1-m}$ the limit $(\ref{eqn:limitLambdas})$, then $\dots\leq t_{-1}\leq t_{0}\leq t_{1}\leq\cdots$. In other words, $\mathbf{t}:=(\dots,t_{-1},t_{0},t_{1},\dots)$ belongs to $\mathfrak{X}$ and $\{\lambda(N)\in\mathbb{GT}_{N}\}_{N\geq 1}$ $\mathbf{b}$-stabilizes to $\mathbf{t}$. We can now apply the first part of the proof (Steps 1–5) to show that the limit $(\ref{mult:limit})$ holds uniformly on compact subsets of $(\mathbb{C}^{*})^{k}$ and we are done. ∎

3.3. Type B-C-D characters: statements of results

Definition 3.10**.**

Let $\mathfrak{Y}$ be the set of nondecreasing, nonnegative integer sequences, i.e.,

[TABLE]

We say that a sequence $\{\lambda(N)\in\mathbb{GT}^{+}_{N}\}_{N\geq 1}$ stabilizes to $\mathbf{y}\in\mathfrak{Y}$ if

[TABLE]

To state our results, we need to define some functions parametrized by elements $\mathbf{y}\in\mathfrak{Y}$, by a type $G\in\{B,C,D\}$, and by a nonnegative integer $m\in\mathbb{N}_{0}$:

[TABLE]

[TABLE]

[TABLE]

In the expressions above, $x^{u}$, $x^{v}$ and $x^{\frac{1}{2}}$ (for $G=B$) stand for $e^{u\ln{x}}$, $e^{v\ln{x}}$ and $e^{\ln{x}/2}$, where $\ln{x}$ is defined over the branch of the logarithm with the cut along the negative real axis. The $u$-contour $\mathcal{L}$ consists of the directed lines $(+\infty-\frac{\pi\mathbf{i}}{2\ln{q}},-\infty-\frac{\pi\mathbf{i}}{2\ln{q}})$ and $(-\infty+\frac{\pi\mathbf{i}}{2\ln{q}},+\infty+\frac{\pi\mathbf{i}}{2\ln{q}})$, see Figure 1.

Lemma 3.11**.**

The functions $\Phi^{\mathbf{y},B}_{m}(x;q)$, $m\in\mathbb{N}_{0}$, defined by $(\ref{typeB})$, are analytic on $\mathbb{C}\setminus((-\infty,0]\cup\{1\}\cup\{q^{n+\frac{1}{2}}:n\in\mathbb{Z}\})$. The functions $\Phi^{\mathbf{y},G}_{m}(x;q)$, $m\in\mathbb{N}_{0}$, $G\in\{C,D\}$, defined by $(\ref{typeC})$, $(\ref{typeD})$, are analytic on $\mathbb{C}\setminus((-\infty,0]\cup\{q^{n}:n\in\mathbb{Z}\})$.

Moreover, all of these functions admit analytic continuations to $\mathbb{C}^{*}$, and satisfy $\Phi_{m}^{\mathbf{y},G}(x;q)=\Phi_{m}^{\mathbf{y},G}(1/x;q)$.

Theorem 3.12**.**

Let $\{\lambda(N)\in\mathbb{GT}^{+}_{N}\}_{N\geq 1}$ be a sequence that stabilizes to some $\mathbf{y}\in\mathfrak{Y}$, and let $m\in\mathbb{N}_{0}$. Then

[TABLE]

holds uniformly on compact subsets of $\mathbb{C}^{*}$.

For any $k\in\mathbb{N}$ and $G\in\{B,C,D\}$, define the constant

[TABLE]

and the functions

[TABLE]

The determinant in $(\ref{def:Phik2})$ vanishes when $x_{i}=x_{j}$ or $x_{i}=1/x_{j}$, for some $i\neq j$. Then the poles of $V^{s}(x_{1},\ldots,x_{k})$ are cancelled out so that the functions $\Phi^{\mathbf{y},G}(x_{1},\ldots,x_{k};q)$ are analytic on $(\mathbb{C}^{*})^{k}$.

Theorem 3.13**.**

Let $k\in\mathbb{N}$, $G\in\{B,C,D\}$, and $\{\lambda(N)\in\mathbb{GT}^{+}_{N}\}_{N\geq 1}$ be a sequence of nonnegative signatures that stabilizes to $\mathbf{y}\in\mathfrak{Y}$. Then

[TABLE]

holds uniformly on compact subsets of $(\mathbb{C}^{*})^{k}$. Conversely, if $\{\lambda(N)\in\mathbb{GT}^{+}_{N}\}_{N\geq 1}$ is a sequence of nonnegative signatures such that the limit in the left hand side of $(\ref{eqn:multiasymptotics2})$ exists and is uniform on compact subsets of $(\mathbb{C}^{*})^{k}$, for any $k\in\mathbb{N}$, then there exists $\mathbf{y}\in\mathfrak{Y}$ such that $\{\lambda(N)\in\mathbb{GT}_{N}\}_{N\geq 1}$ stabilizes to $\mathbf{y}$, and the limit relation $(\ref{eqn:multiasymptotics2})$ holds.

3.4. Integral representations and multivariate formulas for B-C-D type characters

Theorem 3.14**.**

Let $0\leq m\leq N-1$ be integers, $\lambda\in\mathbb{GT}^{+}_{N}$ and $a\in\mathbb{C}$; then

[TABLE]

[TABLE]

[TABLE]

In the right hand sides of $(\ref{thm:typeB})$, $(\ref{thm:typeC})$ and $(\ref{thm:typeD})$, the $z$-contour encloses the smallest closed interval $I\subseteq\mathbb{R}$ that contains all the singularities $\{q^{\pm(\lambda_{i}+N-i+\epsilon)}\}_{1\leq i\leq N}$ (where $\epsilon=\frac{1}{2},1,0$ for types $B,C,D$, respectively), but it does not enclose the origin, whereas the $w$-contour encloses the $z$-contour and the origin.

Remark 3.15**.**

There are some apparent singularities in the right sides of $(\ref{thm:typeB})$, $(\ref{thm:typeC})$ and $(\ref{thm:typeD})$, as functions of $a$, coming from Pochhammer symbols in the denominators. However, the left sides of these identities are entire functions of $a$. Therefore the integrals in the identities vanish at the apparent singularities.

Remark 3.16**.**

If the limit $q\rightarrow 1$, these integral representations recover [GP, Thm. 3.18], for type $C$, and the integral representations coming from [BG, Props. 7.3, 7.4] for types $B$, $D$.

Proof.

The proofs of the three integral representations are very similar. We give full details for the case $G=B$, and leave the cases $G=C,D$ to the reader.

Step 1. From Lemma 2.8 and Proposition 2.9, for any half-integer $a\geq N+\frac{1}{2}$ ($a-\frac{1}{2}\in\mathbb{Z}$), and $0\leq m\leq N-1$, we obtain

[TABLE]

In the next two steps, we find a double integral representation for $\chi^{B}_{(b+1,1^{c},0^{N-c-1})}(y_{1},\ldots,y_{N})$, for $b,c\in\mathbb{N}_{0}$ with $b$ is large enough, $N\geq c+1$, and $y_{1},\ldots,y_{N}>0$. Then we specialize these parameters to obtain a double integral representation for $(\ref{step1})$, when $a$ is very large.

Step 2. Let us work temporarily with any $b,c\in\mathbb{N}_{0}$ such that $b>c+1$, $N\geq c+1$, and any positive real numbers $y_{1},\ldots,y_{N}>0$. Let us denote

[TABLE]

We shall use the following generating functions:

[TABLE]

From Proposition 2.10, and $(\ref{generatingH})$, we obtain

[TABLE]

The sum in brackets in $(\ref{step2:0})$ can be written as two sums, with $c+1$ terms each. The first of those sums is

[TABLE]

To give an integral representation for the second sum, we use:

[TABLE]

The second sum is

[TABLE]

As a result, $\chi^{B}_{(b+1,1^{c})}(y_{1},\ldots,y_{N})$ is the sum of the following two double contour integrals:

[TABLE]

Step 3. Now we rewrite the contour integral representations in $(\ref{step2:1})$ and $(\ref{step2:2})$. Let us begin with $(\ref{step2:1})$. The integrand is of order $O(|v|^{-2})$, as $|v|\rightarrow\infty$. Then we can deform the $v$-contour, pass it through $\infty$, and have the new $v$-contour enclosing the singularities $1,y_{1},y_{1}^{-1},\ldots,y_{N},y_{N}^{-1}$; a minus sign appears in making this deformation. Deform also the $u$-contour and make it very large (in particular, it encloses both [math] and $1$). Next, break the integral $(\ref{step2:1})$ into two integrals by writing $u^{-2N-2}(1-u^{c+1})/(1-u)$ as the difference $u^{-2N-2}/(1-u)-u^{-2N+c-1}/(1-u)$. The first integral, corresponding to $u^{-2N-2}/(1-u)$, is of order $O(|u|^{-2})$, when $|u|\rightarrow\infty$. Thus there is no pole outside the contour, meaning that the first integral vanishes. From these considerations, $(\ref{step2:1})$ equals

[TABLE]

where the $v$-contour encloses $1,y_{1},y_{1}^{-1},\ldots,y_{N},y_{N}^{-1}$, but not the origin, while the $u$-contour encloses both [math] and $1$. After making the change of variables

[TABLE]

we can have the new $z$-contour enclosing the singularities $1,y_{1},\ldots,y_{N}^{-1}$, but not around the origin, whereas the new $w$-contour encloses the $z$-contour and [math]. Then the integral $(\ref{step2:1})$ equals

[TABLE]

For the integral $(\ref{step2:2})$, follow the same procedure. Pass the $v$-contour though infinity so that it encloses the singularities $1,y_{1},\ldots,y_{N}^{-1}$, but not the origin; a negative sign appears. Break the integral into two by using $u^{-c-1}(1-u^{c+1})/(1-u)=u^{-c-1}/(1-u)-1/(1-u)$, and get rid of the second integral because no singularity at [math] remains. Then do the change of variables $(\ref{changevars})$ again. After these simplifications, $(\ref{step2:2})$ equals

[TABLE]

The integrand, as a function of $w$, has only $w=z$ as a singularity. Thus by expanding the $w$-contour, so that it swallows the $z$-contour, we pick up the residue $w=z$. In enlarging the $w$-contour, the total residue that we pick up is (when $w=z$, the residue of the integrand in $(\ref{step3})$ is $-z^{2N+b-c-1}$):

[TABLE]

Thus $(\ref{step3})$ does not change if the $w$-contour in that formula is replaced by one that encloses both the origin and the $z$-contour. We conclude that $\chi^{B}_{(b+1,1^{c},0^{N-c-1})}(y_{1},\ldots,y_{N})$ equals the sum of $(\ref{step3:0})$ and $(\ref{step3})$, the latter of which has the enlarged $w$-contour instead.

Step 4. In step 3, specialized at $b=a-N-\frac{1}{2}$, $c=N-m-1$, and $y_{i}=q^{\lambda_{i}+N-i+\frac{1}{2}}$, we found

[TABLE]

Plugging the above formula into $(\ref{step1})$, and making simple algebraic manipulations, we finally arrive at the desired identity $(\ref{thm:typeB})$. Recall that, in step 1, we assumed that $a$ was a very large positive half-integer. However, observe that both sides of our identity are rational functions of $q^{a}$ (for the right side, this is seen after a residue expansion), so the identity $(\ref{thm:typeB})$ follows for all $a\in\mathbb{C}$. ∎

Recall the Frobenius coordinates for a partition $\lambda$. Let $d$ be the length of the main diagonal of the Young diagram corresponding to $\lambda$. The Frobenius coordinates $(a_{1},\ldots,a_{d}\mid b_{1},\ldots,b_{d})$ of $\lambda$ are

[TABLE]

where $\lambda^{\prime}:=(\lambda_{1}^{\prime},\lambda_{2}^{\prime},\ldots)$ is the conjugate partition of $\lambda$. For example, the “hook-shaped” partition $\lambda=(a+1,1^{b})$ has Frobenius coordinates $(a\mid b)$.

Proposition 3.17** (Frobenius identities).**

Let $N\in\mathbb{N}$, and $\lambda\in\mathbb{GT}^{+}_{N}$ be a partition with Frobenius coordinates $(a_{1},\ldots,a_{d}\mid b_{1},\ldots,b_{d})$. Denote $\chi^{G}_{\mu}=\chi^{G}_{\mu}(x_{1},\ldots,x_{N})$, for any $\mu\in\mathbb{GT}^{+}_{N}$. Also, for any $a,b\in\mathbb{N}_{0}$, $N\geq b+1$, denote $(a|b):=(a+1,1^{b},0^{N-b-1})\in\mathbb{GT}^{+}_{N}$. Then

[TABLE]

Proof.

This is a well-known fact in representation theory, which was derived in [ESK]. The same statement holds for a more general class of functions, called generalised Schur functions, see [SV, Thm 3.1]. That theorem specializes to our Proposition in three special cases, as described in [SV, Section 4.2]. ∎

Theorem 3.18**.**

For any $1\leq k\leq N$, $G\in\{B,C,D\}$, let

[TABLE]

Then for any $\lambda\in\mathbb{GT}^{+}_{N}$, and $x_{1},\ldots,x_{k}\in\mathbb{C}^{*}$, we have

[TABLE]

Proof.

Let us give details for $G=B$ (where $\epsilon=\frac{1}{2}$), leaving the cases $G=C,D$ to the reader. Let $m_{1}\geq\dots\geq m_{k}$ be any integers such that $m_{k}\geq k$. The partition $(m_{1},\ldots,m_{k},k^{N-k})$ has Frobenius coordinates

[TABLE]

By Lemma 2.8 and Proposition 3.17, we have

[TABLE]

By Lemma 2.8 again, we have

[TABLE]

From Proposition 2.9, we obtain

[TABLE]

as well as

[TABLE]

Plugging these formulas into $(\ref{mastereqn})$, and after some algebraic manipulations, we deduce that $(\ref{thm2:typeG})$ holds for $x_{i}=q^{m_{i}+N-i+\frac{1}{2}}$, $i=1,\ldots,k$. Since $m_{1}\geq\dots\geq m_{k}\geq k$ are arbitrary, and both sides of $(\ref{thm2:typeG})$ are rational functions on $x_{1},\ldots,x_{k}$ with the only poles on both sides being $x_{i}=0$, $i=1,\ldots,k$, the identity $(\ref{thm2:typeG})$ holds for all $x_{1},\ldots,x_{k}\in\mathbb{C}^{*}$ too. ∎

3.5. Type B-C-D characters: proofs of results

The following lemma is the analogous to Lemma 3.8; its proof is also very similar and we omit it.

Lemma 3.19**.**

Let $0\leq m\leq N-1$ be integers, $\lambda\in\mathbb{GT}^{+}_{N}$, $G\in\{B,C,D\}$, and $x$ be a complex number in the domain $\mathbb{C}\setminus((-\infty,0]\cup\{1\}\cup\{q^{n+\epsilon}:n\in\mathbb{Z}\})$, such that $q^{N}<|x|<q^{-N}$. Then

[TABLE]

admits the following integral representation (recall $l_{1},\ldots,l_{N}$ are defined in Notation 2.7):

[TABLE]

[TABLE]

[TABLE]

Remark 3.20**.**

The condition $q^{N}<|x|<q^{-N}$ ensures that the double integrals converge.

Proof of Theorems 3.12, 3.13, and Lemma 3.11..

It is not difficult to show that the formulas $(\ref{typeB})$, $(\ref{typeC})$ and $(\ref{typeD})$ define analytic functions on $\mathbb{C}\setminus((-\infty,0]\cup\{1\}\cup\{q^{n+\epsilon}:n\in\mathbb{Z}\})$, proving the first part of Lemma 3.11. Theorem 3.13 is a consequence of Theorems 3.12 and 3.18, thus it suffices to prove Theorem 3.12. We claim that (3.23) holds for $x$ belonging to compact subsets of $\mathbb{C}\setminus((-\infty,0]\cup\{1\}\cup\{q^{n+\epsilon}:n\in\mathbb{Z}\})$. This uses Lemma 3.19 and follows closely the proof of Theorem 3.3 above, so we omit the details. Note that $x$ should avoid $(-\infty,0]$ because $x^{u}$, $x^{v}$ and $x^{\frac{1}{2}}$ are only defined on $\mathbb{C}\setminus(-\infty,0]$; also $x$ should avoid $\{1\}\cup\{q^{n+\epsilon}:n\in\mathbb{Z}\}$ to avoid singularities in the right hand sides of $(\ref{thm:typeB})$, $(\ref{thm:typeC})$ and $(\ref{thm:typeD})$. Also note that, since the Laurent polynomials $\chi_{\lambda(N)}^{G}(q^{\epsilon},\cdots,q^{m-1+\epsilon},x,q^{m+1+\epsilon},\cdots,q^{N-1+\epsilon})$ are invariant with respect to the inversion $x\mapsto x^{-1}$, (3.23) for $x\in\mathbb{C}\setminus\mathbb{R}$ implies $\Phi^{\mathbf{y},G}_{m}(x;q)=\Phi^{\mathbf{y},G}_{m}(1/x;q)$, for $x\in\mathbb{C}\setminus\mathbb{R}$.

We still have to show: (A) $\Phi^{\mathbf{y},G}_{m}(x;q)$ admits an analytic continuation to $\mathbb{C}^{*}$ and the limit $(\ref{eqn:asymptotics2})$ continues to hold uniformly for $x$ in compact subsets of $\mathbb{C}^{*}$; and (B) the converse statement that if the limit in the left side of $(\ref{thm:multivariablesymplectic})$ exists, then $\{\lambda(N)\in\mathbb{GT}^{+}_{N}\}_{N\geq 1}$ stabilizes to some $\mathbf{y}\in\mathfrak{Y}$.

Observe that (A) would immediately imply $\Phi^{\mathbf{y},G}_{m}(x;q)=\Phi^{\mathbf{y},G}_{m}(1/x;q)$, for all $x\in\mathbb{C}^{*}$.

The proof of (B) is identical to the proof of the analogous result of Theorem 3.3, except that it uses the proof of Proposition 6.2 (instead of Proposition 5.2) and the proof of Theorem 6.5 (instead of Theorem 5.6). Let us finish by proving (A).

Let $R>1$ be a (large) real number such that $R\notin\{q^{n+\epsilon}:n\in\mathbb{Z}\}$. We claim that the sequence

[TABLE]

of holomorphic functions on $\mathbb{C}^{*}$ is uniformly bounded on $\{1/R<|z|<R\}$. In fact, from the nonnegativity of the branching coefficients for symplectic/orthogonal characters (see Prop. 2.5), and the fact that $1/R<|z|<R$ implies $|z|^{n}<R^{n}+1/R^{n}$, for any $n\in\mathbb{Z}$, we obtain

[TABLE]

By our choice of $R$, the limit $(\ref{eqn:asymptotics2})$ holds pointwise for $x=R,1/R$, implying that

[TABLE]

are both uniformly bounded sequences, thus implying our claim. Next we can apply Montel’s theorem: it implies that any subsequence of $(\ref{eqn:sequencesymplectic})$ has subsequential limits, which are analytic on $\{1/R<|z|<R\}$. Each such holomorphic function agrees with $\Phi_{m}^{\mathbf{y},G}(z;q)$ on an open set, so by analytic continuation, they must all be the same. Since $R>1$ was arbitrary, the result follows. ∎

4. Branching graphs

4.1. Generalities on Branching Graphs

We recall some general facts about branching graphs and their boundaries. Our exposition is similar to that of [O4]; see also [GO]. Similar statements, in equivalent forms, can be also found in [DF, D, W].

For a Borel space $X$, let $\mathcal{M}(X)$ denote the set of probability measures on $X$. The set $\mathcal{M}(X)$ is a convex subset of the vector space of finite, signed measures on $X$. A Markov kernel between Borel spaces $K:X\dashrightarrow Y$ induces an affine map $K:\mathcal{M}(X)\rightarrow\mathcal{M}(Y)$ between convex sets, that we denote by the same letter.

Let $X_{1},X_{2},\ldots$ be a sequence of countable sets, equipped with their Borel structure coming from the discrete topology. Assume we have, for each $N\geq 1$, a Markov kernel $\Lambda^{N+1}_{N}:X_{N+1}\dashrightarrow X_{N}$. Then the chain of Markov kernels

[TABLE]

naturally induces the chain

[TABLE]

of affine maps of convex sets. Let $\varprojlim{\mathcal{M}(X_{N})}$ be the projective limit. As a set, it consists of coherent systems $\{M_{N}\in\mathcal{M}(X_{N})\}_{N\geq 1}$, i.e., for each $N\geq 1$, the probability measures $M_{N}$ and $M_{N+1}$ are related by

[TABLE]

More explicitly, the coherency property is

[TABLE]

The sequence $\{X_{N},\Lambda^{N+1}_{N}:N\geq 1\}$ is called a branching graph.

Equip $\varprojlim{\mathcal{M}(X_{N})}$ with the Borel structure arising from the embedding

[TABLE]

The product space in $(\ref{qschur:inclusion})$ is convex and $\varprojlim{\mathcal{M}(X_{N})}$ is a convex subset.

Definition 4.1**.**

The set of extreme points of the convex space $\varprojlim{\mathcal{M}(X_{N})}$ is called the (minimal) boundary of the branching graph $\{X_{N},\Lambda^{N+1}_{N}:N\geq 1\}$. Let us denote it by $\Omega$. Given $\omega\in\Omega$, we denote by $\{M_{N}^{\omega}\}_{N\geq 1}$ the corresponding coherent system.

Theorem 4.2** ([O4], Thm. 9.2).**

The set $\Omega$ is a Borel subset of $\varprojlim{\mathcal{M}(\mathbb{GT}_{N})}$. Moreover, for every coherent system $\{M_{N}\}_{N\geq 1}$, there exists a unique Borel probability measure $\pi\in\mathcal{M}(\Omega)$ such that

[TABLE]

Conversely, every $\pi\in\mathcal{M}(\Omega)$ gives a coherent system by the formula above. The resulting map $\mathcal{M}(\Omega)\rightarrow\varprojlim{\mathcal{M}(X_{N})}$ is a bijection.

Let us also define the Martin boundary of the branching graph $\{X_{N},\Lambda^{N+1}_{N}:N\geq 1\}$. If we compose the Markov kernels $\Lambda^{n+1}_{n}:X_{n+1}\dashrightarrow X_{n}$, $N>n\geq k$, we obtain

[TABLE]

In particular, if $N>k$ and $x(N)\in X_{N}$, then $\Lambda^{N}_{k}(x(N),\cdot)$ is a probability measure on $X_{k}$.

Definition 4.3**.**

Let $\{M_{k}\in\mathcal{M}(X_{k})\}_{k\geq 1}$ be a coherent system for which there exists a sequence $\{x(N)\in X_{N}\}_{N\geq 1}$ such that

[TABLE]

The Martin boundary of the branching graph $\{X_{N},\Lambda^{N+1}_{N}:N\geq 1\}$ is defined as the subset of $\varprojlim{\mathcal{M}(X_{N})}$ consisting of the coherent systems $\{M_{k}\in\mathcal{M}(X_{k})\}_{k\geq 1}$ for which such a sequence $\{x(N)\in X_{N}\}_{N\geq 1}$ exists. The Martin boundary is denoted $\Omega^{\textrm{Martin}}$.

Theorem 4.4** ([OO2], Thm 6.1).**

The minimal boundary is contained in the Martin boundary. In other words, for any $\omega\in\Omega$, there exists a sequence $\{x(N)\in X_{N}\}_{N\geq 1}$ such that

[TABLE]

4.2. Symmetric $q$-Gelfand-Tsetlin graph

Let $\sigma=(\sigma_{1},\sigma_{2},\ldots)\in\{+1,-1\}^{\infty}$ be arbitrary; we will construct a branching graph associated to $\sigma$.

For each $N\geq 1$, define the matrix $[\Lambda^{N+1}_{N}(\lambda,\mu)]_{\lambda,\mu}$ of format $\mathbb{GT}_{N+1}\times\mathbb{GT}_{N}$ via the branching relations

[TABLE]

This means the following. For $\lambda\in\mathbb{GT}_{N+1}$, the left hand sides of (4.3) are symmetric Laurent polynomials in $N$ variables. The Schur polynomials $s_{\mu}(x_{1},\ldots,x_{N})$, $\mu\in\mathbb{GT}_{N}$, form a basis of the space of symmetric Laurent polynomials in $x_{1},\ldots,x_{N}$. Thus we are guaranteed the existence and uniqueness of the coefficients $\Lambda^{N+1}_{N}(\lambda,\mu)$, $\mu\in\mathbb{GT}_{N}$, in the right hand sides of (4.3).

From Proposition (2.1) and the homogeneity of Schur polynomials, we obtain

[TABLE]

Lemma 4.5**.**

For each $N\geq 1$, the matrix $[\Lambda^{N+1}_{N}(\lambda,\mu)]$ is stochastic.

Proof.

For any $\lambda\in\mathbb{GT}_{N+1}$, $\mu\in\mathbb{GT}_{N}$, let us show $\Lambda^{N+1}_{N}(\lambda,\mu)\geq 0$. From Proposition $(\ref{prop:evaluation})$, it follows that $s_{\kappa}(1,q,\ldots,q^{K})>0$ if $q>0$. Then $\Lambda^{N+1}_{N}(\lambda,\mu)\geq 0$ follows from the explicit formula $(\ref{Lambda:formulas})$. Next, for any fixed $\lambda\in\mathbb{GT}_{N+1}$, set $x_{i}=q^{i-1}$, $i=1,2,\ldots,N$, in the polynomial equalities $(\ref{eqn:link})$ to show $1=\sum_{\mu}{\Lambda^{N+1}_{N}(\lambda,\mu)}$ for both $\sigma_{N}=1$ and $\sigma_{N}=-1$. ∎

Instead of $\sigma=(\sigma_{1},\sigma_{2},\ldots)\in\{+1,-1\}^{\infty}$, we can consider the sequence $\mathbf{b}=(b(1),b(2),\ldots)\in\mathbb{N}_{0}^{\infty}$ given by

[TABLE]

The sequence $\mathbf{b}$ satisfies $b(1)\in\{0,1\}$ and $b(n+1)-b(n)\in\{0,1\}$ for all $n\geq 1$; conversely any sequence $\mathbf{b}$ with these properties arises from some $\sigma\in\{+1,-1\}^{\infty}$ via the relations $(\ref{sigmab})$.

Definition 4.6**.**

The sequence $\{\mathbb{GT}_{N},\Lambda^{N+1}_{N}:N\geq 1\}$, defined by $(\ref{eqn:link})$, is called the symmetric $q$-Gelfand-Tsetlin graph associated to $\sigma\in\{\pm 1\}^{\infty}$ (or associated to $\mathbf{b}\in\mathbb{N}_{0}^{\infty}$). The minimal boundary of the symmetric $q$-Gelfand-Tsetlin graph associated to $\sigma\in\{\pm 1\}^{\infty}$ will be denoted $\Omega_{q}(\sigma)$, or simply $\Omega_{q}$, if there is no confusion about $\sigma$. Its Martin boundary will be denoted $\Omega^{\textrm{Martin}}_{q}(\sigma)$, or just $\Omega_{q}^{\textrm{Martin}}$.

It will be shown later in Theorems 5.6 and 5.7 that whenever $\sigma,\sigma^{\prime}\in\{\pm 1\}^{\infty}$ satisfy the generic condition (5.1), the topological spaces $\Omega_{q}(\sigma)$ and $\Omega_{q}(\sigma^{\prime})$ are homeomorphic. The same remark applies to the Martin boundaries.

4.3. BC type $q$-Gelfand-Tsetlin graph

For $G=B,C$ and $D$, set $\epsilon=\frac{1}{2},1$, and [math], respectively. For each $N\geq 1$, define the matrix $[\Lambda^{N+1}_{N}(\lambda,\mu)]_{\lambda,\mu}$ of format $\mathbb{GT}^{+}_{N+1}\times\mathbb{GT}^{+}_{N}$ via the branching relation

[TABLE]

From Proposition 2.5, we have

[TABLE]

Note that the kernels $\Lambda^{N+1}_{N}$ depend on $G$, but we suppress it from the notation.

Repeating the proof of Lemma 4.5, we obtain:

Lemma 4.7**.**

For each $N\geq 1$, the matrix $[\Lambda^{N+1}_{N}(\lambda,\mu)]$ is stochastic.

Definition 4.8**.**

The sequence $\{\mathbb{GT}^{+}_{N},\Lambda^{N+1}_{N}:N\geq 1\}$, defined by $(\ref{df:symplecticlinks})$, is called the BC type $q$-Gelfand-Tsetlin graph. The (minimal) boundary of the BC type $q$-Gelfand-Tsetlin graph will be denoted $\Omega_{q}^{G}$. The Martin boundary of the BC type $q$-Gelfand-Tsetlin graph will be denoted $\Omega_{q}^{G,\textrm{Martin}}$.

The boundary $\Omega^{G}_{q}$ depends on the type $G\in\{B,C,D\}$. We show later in Theorems 6.5 and 6.6 that, as topological spaces, all three of them are homeomorphic to each other (but they correspond to three distinct families of coherent systems on the BC type $q$-Gelfand-Tsetlin graph). The same remark applies to the Martin boundaries $\Omega_{q}^{G,\textrm{Martin}}$.

5. Boundary of the symmetric $q$-Gelfand-Tsetlin graph

In this section, let us fix a sequence $\sigma=(\sigma_{1},\sigma_{2},\ldots)\in\{\pm 1\}^{\infty}$ satisfying the assumption

[TABLE]

Equivalently, in terms of the sequence $\mathbf{b}$ given by $(\ref{sigmab})$,

[TABLE]

The goal of this section is to characterize the minimal boundary of the symmetric $q$-Gelfand-Tsetlin graph associated to $\mathbf{b}$. To proceed via the ergodic method of Vershik-Kerov, [VK1, V], we need to first characterize the Martin boundary of the symmetric $q$-Gelfand-Tsetlin graph.

In subsections 5.1, 5.2 and 5.3, we identify the Martin boundary with the set $\mathfrak{X}$ of all doubly infinite, nondecreasing integer sequences (recall Definition 3.1). In Subsection 5.4, we show that the minimal boundary coincides with the Martin boundary. The proof of the latter statement is based on the Law of Large Numbers of Theorem 5.5.

5.1. A family of coherent probability measures

In this section, we define a map from the set $\mathfrak{X}$ into the Martin boundary of the symmetric $q$-Gelfand-Tsetlin graph.

Let $k\in\mathbb{N}$ and let $C_{k}$ be the space of analytic functions $f(z_{1},\ldots,z_{k})$ on $(\mathbb{C}^{*})^{k}$ for which there exist $a_{m_{1},\ldots,m_{k}}\in\mathbb{C}$, $m_{1},\ldots,m_{k}\in\mathbb{Z}$, such that

[TABLE]

and the right side is an absolutely and uniformly convergent sum on compact subsets of $(\mathbb{C}^{*})^{k}$. Then necessarily

[TABLE]

Let us call $C_{k}^{S_{k}}$ the linear subspace of $C_{k}$, consisting of functions $f(z_{1},\ldots,z_{k})$ for which the coefficients in the expansion $(\ref{eqn:fourierexample})$ satisfy

[TABLE]

For example, finite linear combinations of Schur polynomials $s_{\lambda}(z_{1},\ldots,z_{k})$, $\lambda\in\mathbb{GT}_{k}$, are elements of $C_{k}^{S_{k}}$. For each $\mu\in\mathbb{GT}_{k}$, define the linear functional $F_{\mu}:C_{k}^{S_{k}}\rightarrow\mathbb{C}$ by

[TABLE]

Lemma 5.1**.**

The linear functionals $\{F_{\mu}:C_{k}^{S_{k}}\rightarrow\mathbb{C}\}_{\mu\in\mathbb{GT}_{k}}$, defined by $(\ref{def:Fmufunctional})$, satisfy

1. (1)

$F_{\mu}(s_{\nu}(z_{1},\ldots,z_{k}))=\delta_{\mu,\nu}$, for all $\mu,\nu\in\mathbb{GT}_{k}$. 2. (2)

If $\{f_{N}\}_{N\geq 1}\subset C_{k}^{S_{k}}$ is such that $f_{N}\rightarrow f$ uniformly on compact subsets of $(\mathbb{C}^{*})^{k}$, then $f\in C_{k}^{S_{k}}$ and $F_{\mu}(f_{N})\rightarrow F_{\mu}(f)$ for all $\mu\in\mathbb{GT}_{k}$. 3. (3)

Let $h\in C_{k}^{S_{k}}$ be such that $F_{\mu}(h)=0$, for all $\mu\in\mathbb{GT}_{k}$. Then $h$ is identically zero.

Proof.

The first item of the lemma is well known, see for instance [M, I.4, VI.9].

The second item is obvious.

For the third item, let $h\in C_{k}^{S_{k}}$ be such that $F_{\mu}(h)=0$, for all $\mu\in\mathbb{GT}_{k}$. The span of the Schur polynomials $\{s_{\mu}(z_{1},\ldots,z_{k})\}_{\mu\in\mathbb{GT}_{k}}$ is the linear space of symmetric Laurent polynomials on $k$ variables, and this is dense in the space of continuous, symmetric functions on the $k$–dimensional torus $\mathbb{T}^{k}$. Therefore

[TABLE]

for any continuous, symmetric function $g$ on $\mathbb{T}^{k}$. Since $h$ is analytic on $(\mathbb{C}^{*})^{k}$, it is continuous on $\mathbb{T}^{k}$. It follows from (5.4) that $h$ vanishes on $\mathbb{T}^{k}$ and, from its analyticity, $h$ also vanishes on $(\mathbb{C}^{*})^{k}$. ∎

Proposition 5.2**.**

Take any $\mathbf{t}\in\mathfrak{X}$ and let $\{\lambda(N)\in\mathbb{GT}_{N}\}_{N\geq 1}$ be a sequence that $\mathbf{b}$-stabilizes to $\mathbf{t}$. For each $k\geq 1$, $\mu\in\mathbb{GT}_{k}$, the limit

[TABLE]

exists and does not depend on the choice of $\{\lambda(N)\}_{N\geq 1}$. Moreover, $M_{k}^{\mathbf{t}}$ is a probability measure on $\mathbb{GT}_{k}$, and $\{M_{k}^{\mathbf{t}}\}_{k\geq 1}$ is a coherent system.

Proof.

Let $\{\lambda(N)\in\mathbb{GT}_{N}\}_{N\geq 1}$ be any sequence of signatures that $\mathbf{b}$-stabilizes to $\mathbf{t}$. Let $k\geq 1$ be arbitrary, and define $b_{k}(N):=\max\{0,\ b(N)-b(k)\}$, so that $b_{k}(N)=\#\{k<i\leq N:\sigma_{i}=-1\}$ whenever $N>k$. Consider the equality

[TABLE]

which follows from $(\ref{eqn:link})$, by induction on $N-k$. Let us look at the first line of $(\ref{branch1})$. It is clear that $0\leq b(N)-b_{k}(N)\leq k$ and thus

[TABLE]

Moreover if we let

[TABLE]

then the sequence $\{\lambda(N)\}_{N\geq 1}$ $\mathbf{b}_{k}$-stabilizes to $\mathbf{t}(k):=(\ldots,t_{-1}^{k},t_{0}^{k},t^{k}_{1},\ldots)$, $t_{i}^{k}:=t_{i+b(k)}$, for all $i\in\mathbb{Z}$. Then, from the limit (3.3) in Theorem 3.3, the first line of $(\ref{branch1})$ converges to $\Phi^{\mathbf{t}(k)}(x_{1},\ldots,x_{k};q)$ uniformly on compact subsets of $(\mathbb{C}^{*})^{k}$, as $N$ goes to infinity. Then by $(\ref{branch1})$ and Lemma 5.1 (2), the following limit exists:

[TABLE]

We show that $\{a_{\mu}:\mu\in\mathbb{GT}_{k}\}$ determines a probability measure on $\mathbb{GT}_{k}$. Since $\Lambda^{N}_{k}(\lambda(N),\mu)\geq 0$ and $\sum_{\mu\in\mathbb{GT}_{k}}{\Lambda^{N}_{k}(\lambda(N),\mu)}=1$ for all $N>k$, then

[TABLE]

We need an extra argument to show $\sum_{\mu\in\mathbb{GT}_{k}}{a_{\mu}}=1$. Consider the function

[TABLE]

Since $(\ref{branch1})$ converges uniformly on compact subsets, then in particular, it converges pointwise at the point $(x_{1},x_{2},\ldots,x_{k})=(t,tq,\ldots,tq^{k-1})$, for any $t>0$. But at this point, each summand

[TABLE]

is nonnegative. Then the sum $\sum_{\mu\in\mathbb{GT}_{k}}{a_{\mu}t^{|\mu|}}$ must be convergent, for any $t>0$. Let $R>1$ be arbitrary; for any $(z_{1},\ldots,z_{k})\in(\mathbb{C}^{*})^{k}$ such that $Rq^{i-1}<|z_{i}|<R^{-1}q^{i-1}$, the nonnegativity of the branching coefficients for Schur polynomials (see Proposition 2.1) and homogeneity of Schur polynomials imply

[TABLE]

As $R>1$ is arbitrary, we conclude that the sum defining $g$ is absolutely and uniformly convergent on compact subsets of $(\mathbb{C}^{*})^{k}$, and $g$ is a well-defined analytic function in that domain. Moreover, $g\in C_{k}^{S_{k}}$ because each $s_{\mu}$ is a symmetric polynomial.

From items (1)–(2) of Lemma 5.1, we have $F_{\mu}(g)=a_{\mu}/s_{\mu}(1,q,\ldots,q^{k-1})$, for all $\mu\in\mathbb{GT}_{k}$. But we also know $F_{\mu}(\Phi^{\mathbf{t}(k)})=a_{\mu}/s_{\mu}(1,q,\ldots,q^{k-1})$. Therefore $h:=g-\Phi^{\mathbf{t}(k)}\in C_{k}^{S_{k}}$ is such that $F_{\mu}(h)=F_{\mu}(g)-F_{\mu}(\Phi^{\mathbf{t}(k)})=0$, for all $\mu\in\mathbb{GT}_{k}$. Item (3) of Lemma 5.1 then implies $h=0$, i.e., $g=\Phi^{\mathbf{t}(k)}$. In particular, evaluating this equality at the point $(1,q,\ldots,q^{k-1})$ and using the limit (3.3) of Theorem 3.3 yields (below $\{\lambda(N)\}_{N\geq 1}$ is any sequence that $\mathbf{b}$-stabilizes to $\mathbf{t}$):

[TABLE]

Thus if we let $M^{\mathbf{t}}_{k}(\mu):=a_{\mu}$, for any $\mu\in\mathbb{GT}_{k}$, we have that $M^{\mathbf{t}}_{k}$ is a probability measure on $\mathbb{GT}_{k}$. Moreover, because $\Phi^{\mathbf{t}(k)}$ does not depend on the choice of $\mathbf{b}$-stabilizing sequence $\{\lambda(N)\}_{N\geq 1}$, neither does each $a_{\mu}=s_{\mu}(1,q,\ldots,q^{k-1})F_{\mu}(\Phi^{\mathbf{t}(k)})$. The coherency is immediate from the definitions. ∎

5.2. Concentration bound

We keep the notations from the previous subsection.

Proposition 5.3**.**

Let $k\in\mathbb{N}_{0}$ and $\{\lambda(N)\in\mathbb{GT}_{N}\}_{N\geq 1}$ be arbitrary. Further, let $\{\mu(N)\in\mathbb{GT}_{N}\}_{N\geq 1}$ be random variables such that each $\mu(N)$ is distributed according to the probability distribution $\Lambda^{N+1}_{N}(\lambda(N+1),\cdot)$. Then there exists a constant $c>0$, independent of $N$, such that

[TABLE]

We need some preparations for the proof. Given $M\in\mathbb{N}$, $\nu\in\mathbb{GT}_{M+1}$, define two probability distributions on $\mathbb{GT}_{M}$, to be denoted $P^{+}(\cdot|\nu)$ and $P^{-}(\cdot|\nu)$, and given by

[TABLE]

There is an explicit closed formula for the evaluation of a Schur polynomial on a $q$-geometric series, see Proposition 2.3; it gives

[TABLE]

Lemma 5.4**.**

Let $m\in\mathbb{N}$ be fixed. There exists a constant $c^{\prime}>0$ such that, for sufficiently large $M\in\mathbb{N}$, any $\nu\in\mathbb{GT}_{M+1}$ and any integer $1\leq i\leq M-m+1$, the following holds:

Let $\kappa\in\mathbb{GT}_{M}$ be distributed according to $P^{+}(\cdot\mid\nu)$. Then

[TABLE]

Proof.

The proof is a generalization of the proof of [GO, Lem. 3.13].

From Boole’s inequality, it is clear that we only need to consider the case $m=1$. In other words, take any $1\leq i\leq M$ and let us find a constant $c^{\prime}>0$ such that

[TABLE]

Set $a:=\nu_{i}$, $b:=\nu_{i+1}$, so that $a\geq\kappa_{i}\geq b$ almost surely. If $a=b$, then $\textrm{Prob}\left(\kappa_{i}=b=\nu_{i+1}\right)=1$ and there is nothing to prove. Then assume $a\geq b+1$, so there exist values $m\in\mathbb{Z}$ such that $a-b\geq m\geq 1$. For each such value, we estimate the ratio

[TABLE]

and show that it is of order $O(q^{mi})$. Let us actually estimate the ratio $(\ref{eqn:ratio})$, where both probabilities are conditional on given values $\kappa_{1},\ldots,\kappa_{i-1},\kappa_{i+1},\ldots,\kappa_{M}$, which are fixed, but arbitrary, and satisfy $\nu_{j}\geq\kappa_{j}\geq\nu_{j+1}$ for all relevant $j$. Under such conditional probability, and from the formula $(\ref{Pminus})$ for $P^{+}(\cdot|\nu)$, we have

[TABLE]

for any $a\geq d\geq b$, where the hidden constant is independent of $d$. Then for any $a-b\geq m\geq 1$, and using $q\in(0,1)$, we obtain

[TABLE]

Next, since $\kappa_{i}-i$ is strictly decreasing on $i$ and $1+b\leq a=\nu_{i}\leq\kappa_{i-1}$:

[TABLE]

and thus

[TABLE]

Similarly, we can deduce $1\leq b+1-\kappa_{i+1}<b+2-\kappa_{i+2}<\ldots<b+M-i-\kappa_{M}$ and

[TABLE]

From the bounds $(\ref{bound1})$, $(\ref{bound2})$ and $(\ref{bound3})$, we have

[TABLE]

By adding the inequalities $(\ref{expbound})$ over $1\leq m\leq a-b$, and using $1+q+\ldots+q^{a-b-1}<1/(1-q)$:

[TABLE]

Since $\textrm{Prob}(\kappa_{i}\geq b+1)=1-\textrm{Prob}(\kappa_{i}=b)$, we finally obtain

[TABLE]

with positive constant $c^{\prime}:=1/(q;q)_{\infty}^{2}$. Note that the bound just proven was uniform over $\kappa_{1},\ldots,\kappa_{i-1},\kappa_{i+1},\ldots,\kappa_{M}$ that we were conditioning over. Thus the bound also holds without the conditioning, and we are done. ∎

Proof of Proposition 5.3.

If $\sigma_{N+1}=-1$, note that the probability measures $\Lambda^{N+1}_{N}(\lambda(N+1),\cdot)$ and $P^{+}(\cdot|\lambda(N+1))$ are the same. We can apply Lemma 5.4 to show that if $\mu(N)$ is $\Lambda^{N+1}_{N}(\lambda(N+1),\cdot)$-distributed, then

[TABLE]

for some $c>0$ independent of $N$. As $\sigma_{N+1}=-1$, then $b(N+1)=b(N)+1$, so the left side of $(\ref{event})$ equals

[TABLE]

giving us the desired result.

Let us proceed to the case $\sigma_{N+1}=+1$, which will be deduced from the case $\sigma_{N+1}=-1$. From the determinantal definition of Schur polynomials, we have

[TABLE]

where $\kappa^{-}:=(-\kappa_{M},\ldots,-\kappa_{1})\in\mathbb{GT}_{M}$. Therefore $P^{-}(\kappa|\nu):=P^{+}(\kappa^{-}|\nu^{-})$, for any $\kappa\in\mathbb{GT}_{M}$, $\nu\in\mathbb{GT}_{M+1}$. In other words, if $\kappa$ is distributed according to $P^{-}(\cdot|\nu)$, then $\kappa^{-}$ is distributed according to $P^{+}(\cdot|\nu^{-})$.

Next, let $\mu(N)^{-}:=(-\mu(N)_{N},\ldots,-\mu(N)_{1})$, $\lambda(N+1)^{-}:=(-\lambda(N+1)_{N+1},\ldots,-\lambda(N+1)_{1})$; if $c>0$ is the constant for the case $\sigma_{N+1}=-1$ treated above, then

[TABLE]

where the last equality comes from $b(N+1)=b(N)$, in the case $\sigma_{N+1}=+1$. ∎

5.3. Law of Large Numbers and the Martin boundary

Theorem 5.5**.**

Let $\{\mu(N)\in\mathbb{GT}_{N}\}_{N\geq 1}$ be a sequence of random signatures such that each $\mu(N)$ is $M_{N}^{\mathbf{t}}$-distributed (see Proposition 5.2 for the definition of the probability measures $M_{N}^{\mathbf{t}}$). Then, for each $k\in\mathbb{Z}$, the probability of the event

[TABLE]

tends to $1$, as $L$ goes to infinity.

Proof.

Let $\{\lambda(N)\in\mathbb{GT}_{N}\}_{N\geq 1}$ be a sequence of signatures that $\mathbf{b}$-stabilizes to $\mathbf{t}$. From Proposition 5.2, the probability measures $\Lambda^{N}_{K}(\lambda(N),\cdot)$ converge weakly to $M_{K}^{\mathbf{t}}$, for all $K\geq 1$. So if we let $\{\mu^{N,K}\}_{N\geq K}\subset\mathbb{GT}_{K}$ be a sequence of random signatures with $\mu^{N,K}$ being $\Lambda^{N}_{K}(\lambda(N),\cdot)$-distributed, and let $\mu^{K}\in\mathbb{GT}_{K}$ be random $M^{\mathbf{t}}_{K}$-distributed, then $\mu^{N,K}$ converges weakly to $\mu^{K}$.

Let $\epsilon>0$ be arbitrary. Let $c>0$ be the constant given in Proposition 5.3 for the given $k\in\mathbb{N}$ in the statement of the Theorem. There exists $N_{0}\in\mathbb{N}$ large enough so that

[TABLE]

As $\lim_{N\rightarrow\infty}b(N)=\lim_{N\rightarrow\infty}(N-b(N))=+\infty$, there exists a large enough $L\in\mathbb{N}$ such that

[TABLE]

Then Proposition 5.3 implies

[TABLE]

where

[TABLE]

By our choice of $L$, $x(i)\geq N_{0}$ for all $i\geq L_{0}$. Also, the sequence $(x(L),x(L+1),\dots)$ does not contain the same number more than twice. As a result,

[TABLE]

Now since $\lambda(N)$ $\mathbf{b}$-stabilizes to $\mathbf{t}$, we have $\lambda(N)_{b(N)+k}=t_{1-k}$, for large enough $N\in\mathbb{N}$. Also, as mentioned above, $\mu^{N,L}$ converges weakly to $\mu^{L}$, as $N$ goes to infinity. Therefore, from $(\ref{eqn:ineq1})$ and these observations, we obtain the inequality

[TABLE]

Since $\epsilon>0$ was arbitrary, $(\ref{eqn:ineq2})$ implies the theorem. ∎

Theorem 5.6**.**

The Martin boundary of the symmetric $q$-Gelfand-Tsetlin graph associated to $\sigma$ is in bijection with the set $\mathfrak{X}$, under the map $\mathbf{t}\mapsto\{M_{N}^{\mathbf{t}}\}_{N\geq 1}$ of Proposition 5.2.

Proof.

By Proposition 5.2 and Theorem 5.5, each coherent system $\{M_{N}^{\mathbf{t}}\}_{N\geq 1}$ belongs to the Martin boundary, and all of them are distinct. It remains to show that there are no other points in the Martin boundary.

Let $\{M_{k}\}_{k\geq 1}$ be an element of $\Omega_{q}^{\textrm{Martin}}$; then there exists a sequence $\{\lambda(N)\in\mathbb{GT}_{N}\}_{N\geq 1}$ such that

[TABLE]

Let $m\in\mathbb{Z}$ be fixed, but arbitrary, and let $c>0$ be the constant in Proposition 5.3 for $k=|m|+1$. Let $N_{0}\in\mathbb{N}$ be such that

[TABLE]

Since $\lim_{N\rightarrow\infty}b(N)=\lim_{N\rightarrow\infty}(N-b(N))=+\infty$, there exists $L\in\mathbb{N}$ such that $\min\{b(N),N-b(N)\}\geq N_{0}$, for all $N\geq L$. Fix $L$ for the moment. For any $N\geq L$, let $\mu^{N}\in\mathbb{GT}_{L}$ be a random signature which is $\Lambda^{N}_{L}(\lambda(N),\cdot)$-distributed. Also let $\xi^{N}:=\mu^{N}_{b(L)+m}\in\mathbb{Z}$, so it is a random integer. From Proposition 5.3,

[TABLE]

where $x(i):=b(i)$, if $\sigma_{i+1}=-1$ and $x(i):=i-b(i)$, if $\sigma_{i+1}=+1$. By our choice of $L$, $x(i)\geq N_{0}$ for $i\geq L$. Moreover, the sequence $(x(L),x(L+1),\ldots)$ does not contain the same number more than twice. As a result,

[TABLE]

By assumption, $M_{L}$ is the weak limit of $\Lambda^{N}_{L}(\lambda(N),\cdot)$, as $N$ tends to infinity. Therefore, if we let $M$ be the pushforward of $M_{L}$ from $\mathbb{GT}_{L}$ to the coordinate $b(N)+m$, then $M$ is a probability measure and the weak limit of the laws of the random variables $\xi^{N}$. From $(\ref{eqn:boundxi})$, we deduce that $M(\{p\})>2/3$, for any $p\in\mathbb{Z}$ which is a subsequential limit of the sequence $\{\lambda(N)_{b(N)+m}\}_{N\geq 1}$. This cannot occur for more than one $p\in\mathbb{Z}$, or the total measure of $M$ would be at least than $4/3$. On the other hand, there has to be at least one subsequential limit of $\{\lambda(N)_{b(N)+m}\}_{N\geq 1}$, since otherwise the total measure of $M$ would be at most $1/3$. Therefore the sequence $\{\lambda(N)_{b(N)+m}\}_{N\geq 1}$ converges as $N$ goes to infinity. Since $m\in\mathbb{Z}$ was arbitrary, the limit

[TABLE]

exists for any $m\in\mathbb{Z}$. This implies that $\{\lambda(N)\}_{N\geq 1}$ must $\mathbf{b}$-stabilize to some $\mathbf{t}\in\mathfrak{X}$. From $(\ref{eqn:schurdfnMartin})$ and Proposition 5.2, it follows that $M_{k}=M_{k}^{\mathbf{t}}$ for any $k\in\mathbb{N}$, i.e., $\{M_{k}\}_{k}$ coincides with the coherent system $\{M_{N}^{\mathbf{t}}\}_{N\geq 1}$, concluding the proof. ∎

5.4. Characterization of the minimal boundary

Theorem 5.7**.**

The minimal boundary $\Omega_{q}$ of the symmetric $q$-Gelfand-Tsetlin graph associated to $\sigma$ is equal to the Martin boundary $\Omega_{q}^{\textrm{Martin}}$.

Proof.

This is a Corollary of Theorems 5.5, 5.6 and 4.4. Indeed, by the latter one, the minimal boundary is contained in the Martin boundary and it remains to show that the coherent systems $\{M_{N}^{\mathbf{t}}\}_{N\geq 1}$ are extreme points of the convex space $\lim_{\leftarrow}{\mathcal{M}(\mathbb{GT}_{N})}$ of coherent systems on the symmetric $q$-Gelfand-Tsetlin graph. By Theorem 5.5, all the measures $\{M_{N}^{\mathbf{t}}\}_{N\geq 1}$ have pairwise disjoint supports, hence one cannot be a convex combination of the others, which finishes the proof. See also [GO, proof of Thm. 3.12] and [O5, Step 3 in proof of Thm. 6.2] for more detailed expositions of similar proofs. ∎

6. Boundary of the BC type $q$-Gelfand-Tsetlin graph

In this section, we characterize the Martin and minimal boundaries of the BC type $q$-Gelfand-Tsetlin graph. We suppress the type $G\in\{B,C,D\}$ from most notations, for simplicity. The parameter $\epsilon$ is $\frac{1}{2},1,0$, for $G=B,C,D$, respectively.

The reader should compare Section 5 with this one. Both follow the same approach and yield similar results.

6.1. A family of coherent probability measures

In Proposition 6.2, we define a map from $\mathfrak{Y}$ into construct the measures of the Martin boundary of the BC type $q$-Gelfand-Tsetlin graph.

Let $k\in\mathbb{N}$ be arbitrary. Recall the linear space $C_{k}$, defined at the beginning of Section 5.1, of analytic functions $f(z_{1},\ldots,z_{k})$ on $(\mathbb{C}^{*})^{k}$ for which an absolutely and uniformly convergent expansion

[TABLE]

exists. Let us call $C_{k}^{W_{k}}$ the linear subspace of $C_{k}$, consisting of functions $f(z_{1},\ldots,z_{k})$ for which the coefficients in the expansion $(\ref{eqn:fourierbc})$ satisfy

[TABLE]

For example, finite linear combinations of the symplectic/orthogonal polynomials $\chi^{G}_{\lambda}(z_{1},\ldots,z_{k})$, $G\in\{B,C,D\}$, $\lambda\in\mathbb{GT}^{+}_{k}$, belong to $C_{k}^{W_{k}}$. For each $G\in\{B,C,D\}$ and $\mu\in\mathbb{GT}^{+}_{k}$, define the functionals $F_{\mu}^{G}:C_{k}^{W_{k}}\rightarrow\mathbb{C}$ via

[TABLE]

where

[TABLE]

[TABLE]

There is an explicit formula for $\widetilde{F}_{\mu}^{G}(\chi_{\mu}^{G}(z_{1},\ldots,z_{k}))$, see [OlOs, Lem. 3.5], which shows that it is nonzero and therefore the definition of $F_{\mu}^{G}$ is well-posed.

Lemma 6.1**.**

The linear functionals $\{F_{\mu}^{G}:C_{k}^{W_{k}}\rightarrow\mathbb{C}\}_{\mu\in\mathbb{GT}_{k}}$ satisfy

1. (1)

$F^{G}_{\mu}(\chi^{G}_{\nu}(z_{1},\ldots,z_{k}))=\delta_{\mu,\nu}$, for all $\mu,\nu\in\mathbb{GT}^{+}_{k}$. 2. (2)

If $\{f_{N}\}_{N\geq 1}\subset C_{k}^{W_{k}}$ is such that $f_{N}\rightarrow f$ uniformly on compact subsets of $(\mathbb{C}^{*})^{k}$, then $f\in C_{k}^{W_{k}}$ and $F_{\mu}^{G}(f_{N})\rightarrow F_{\mu}^{G}(f)$ for all $\mu\in\mathbb{GT}^{+}_{k}$. 3. (3)

If $h\in C_{k}^{W_{k}}$ and $F_{\mu}^{G}(h)=0$, for all $\mu\in\mathbb{GT}^{+}_{k}$, then $h$ is identically zero.

Proof.

The first item of the lemma is well known, see for instance [OlOs]. The second item is obvious. As for the third item, its proof is similar to the proof of Lemma 5.1 (3). The only difference is that now we need the fact that the linear space of $W_{k}$–symmetric Laurent polynomials on $k$ variables is dense in the space of continuous, $W_{k}$–symmetric functions on the $k$-dimensional torus $\mathbb{T}^{k}$. Indeed note that, under the change of variables $z\in\mathbb{T}\rightarrow[-1,1]\ni x:=(z+z^{-1})/2$, the $W_{k}$–symmetric Laurent polynomials become usual symmetric polynomials. Our statement is then equivalent to the fact that symmetric polynomials are dense on the space of continuous, symmetric functions on $[-1,1]^{k}$, and this follows from Weierstrass approximation theorem. ∎

Proposition 6.2**.**

Take any $\mathbf{y}\in\mathfrak{Y}$. Let $\{\lambda(N)\in\mathbb{GT}^{+}_{N}\}_{N\geq 1}$ be a sequence that stabilizes to $\mathbf{y}$. Then for each $k\geq 1$, $\mu\in\mathbb{GT}^{+}_{k}$, the limit

[TABLE]

exists and does not depend on the choice of $\{\lambda(N)\}_{N\geq 1}$. For each $k\geq 1$, $M_{k}^{\mathbf{y}}$ is a probability measure on $\mathbb{GT}^{+}_{k}$ and, moreover, $\{M^{\mathbf{y}}_{k}\in\mathcal{M}(\mathbb{GT}^{+}_{k})\}_{k\geq 1}$ is a coherent system.

Proof.

Let $\{\lambda(N)\in\mathbb{GT}^{+}_{N}\}_{N\geq 1}$ be any sequence of nonnegative signatures that stabilizes to $\mathbf{y}\in\mathfrak{Y}$; also let $k\in\mathbb{N}$ be arbitrary. From the definition $(\ref{df:symplecticlinks})$ of the Markov kernels $\Lambda^{N+1}_{N}$, we have

[TABLE]

By virtue of Theorem 3.13, the left side of $(\ref{eqn:prelimitdefining})$ converges to the analytic function $\Phi^{\mathbf{y},G}(x_{1},\ldots,x_{k};q)$, on compact subsets of $(\mathbb{C}^{*})^{k}$. The rest of the proof is similar to that of Proposition 5.2. One shows that $\{\Lambda^{N}_{k}(\lambda(N),\mu)\}_{N\geq k}$ has a limit, for any $\mu\in\mathbb{GT}^{+}_{k}$, and moreover each limit

[TABLE]

does not depend on the stabilizing sequence of nonnegative signatures $\{\lambda(N)\}_{N\geq 1}$. One then shows $a_{\mu}\geq 0$ and $\sum_{\mu\in\mathbb{GT}^{+}_{k}}{a_{\mu}}=1$, and therefore the measure $M^{\mathbf{y}}_{k}\in\mathcal{M}(\mathbb{GT}^{+}_{k})$, $M^{\mathbf{y}}_{k}(\mu):=a_{\mu}$, satisfies the desired properties. The coherency of the sequence $\{M^{\mathbf{y}}_{k}\}_{k\geq 1}$ is evident. In working out these outlined steps, one should follow the proof of Proposition 5.2 with the only difference that Lemma 6.1 is used instead of Lemma 5.1. ∎

6.2. Concentration bound

Proposition 6.3**.**

Let $k\in\mathbb{N}$ and let $\{\lambda(N)\in\mathbb{GT}^{+}_{N}\}_{N\geq 1}$ be an arbitrary sequence. Let $\mu(N)\in\mathbb{GT}^{+}_{N}$ be random variables such that each $\mu(N)$ is distributed according to the probability distribution $\Lambda^{N+1}_{N}(\lambda(N+1),\cdot)$. Then there exists a constant $c>0$, independent of $N$, such that

[TABLE]

Proof.

By Boole’s inequality, it suffices to show that for a fixed integer $m\geq 1$, there exists a constant $c>0$, independent of $N$, such that

[TABLE]

Let us do the case $m=1$ and then remark on the small differences needed for the argument in the general case $m\geq 2$. For $m=1$, it will suffice to show the existence of constants $c_{1},c_{2}>0$ such that

[TABLE]

Denote $\lambda(N+1)$ by $\lambda$, and $\mu(N)$ by $\mu$.

We begin with $(\ref{firstline})$. If $\lambda_{N+1}=0$, then $\textrm{Prob}(\mu_{N}\leq\lambda_{N+1}-1)=0$ because $\mu_{N}\geq 0$ almost surely, so assume $\lambda_{N+1}\geq 1$. For each $0\leq b<\lambda_{N+1}$, we want to bound the ratio

[TABLE]

and show it is of order $O(q^{N})$. It is convenient to define a measure $M_{N}(\cdot|\lambda)$ on $\mathbb{GT}^{+}_{N+1}\times\mathbb{GT}^{+}_{N}$ via

[TABLE]

The formula for $\tau^{G}(q^{N+\epsilon};\lambda,\nu,\mu)$ is in the statement of Proposition 2.5. Using Proposition 2.9, and the fact that $0<q<1$, we deduce that $M_{N}(\cdot|\lambda)$ is a positive measure. From Proposition 2.5, we also deduce that the pushforward of $M_{N}(\cdot|\lambda)$, under the projection $\pi:\mathbb{GT}^{+}_{N+1}\times\mathbb{GT}^{+}_{N}\rightarrow\mathbb{GT}^{+}_{N}$, is $\Lambda^{N+1}_{N}(\lambda,\cdot)$. In particular, $M_{N}(\cdot|\lambda)$ is a probability measure. So instead of estimating the ratio $(\ref{ratiosmall})$ with respect to the probability measure $\Lambda^{N+1}_{N}(\lambda,\cdot)$, we estimate it with respect to the probability measure $M_{N}(\cdot|\lambda)$ and show that it is of order $O(q^{N})$.

We can do a further simplification. Instead of considering $M_{N}(\cdot|\lambda)$, let us consider arbitrary nonnegative integers $\nu_{1},\ldots,\nu_{N}$ and $\mu_{1},\ldots,\mu_{N-1}$, such that $\lambda_{i}\geq\nu_{i}\geq\lambda_{i+1}$, $\nu_{j}\geq\mu_{j}\geq\nu_{j+1}$, for relevant $i,j$, and let us estimate the ratio $(\ref{ratiosmall})$ with respect to $M_{N}(\cdot|\lambda)$ conditioned on $\nu_{1},\ldots,\nu_{N}$, $\mu_{1},\ldots,\mu_{N-1}$. For any $0\leq e\leq d\leq\lambda_{N+1}$ (and with respect to this probability measure), we get

[TABLE]

the proportionality constant in the last equation does not depend on $d,e$.

If $G=B$, $\tau^{B}(q^{N+\epsilon};\lambda,\nu,\mu)=1+q^{-N-\epsilon}=1+q^{-N-\frac{1}{2}}$, unless $\nu_{N+1}=0$ in which case $\tau^{B}(q^{N+\epsilon};\lambda,\nu,\mu)=1$. Moreover,

[TABLE]

As a result, we obtain

[TABLE]

and therefore

[TABLE]

If $G=C$, $\tau^{C}=1$ always, and

[TABLE]

As a result, we have

[TABLE]

If $G=D$, $\tau^{D}=0$ unless $\nu_{N+1}\in\{0,\mu_{N}\}$ (at least in this case that we assume $\mu_{N}\leq\lambda_{N+1}$) and $\tau^{D}=1$ in those two cases. Moreover,

[TABLE]

As a result, we have

[TABLE]

Then, for any $0\leq b<\lambda_{N+1}$, the ratio $\textrm{Prob}(\mu_{N}=b)/\textrm{Prob}(\mu_{N}=b+1)$ equals

[TABLE]

By following the same analysis as in the proof of Lemma 5.4, we can upper bound all three expressions above by (the exponentially small) $2q^{N}/(q;q)_{\infty}^{2}$. Let us give, as an example, the details for the type $G=B$. Use the bounds $0<1-q^{b+\frac{1}{2}}<1-q^{b+\frac{3}{2}}$, $1-q^{(2N+1)(b+1)}<1-q^{(2N+1)(b+2)}$ and $(1-q^{\mu_{i}+N-i-b})(1-q^{\mu_{i}+N-i+b+1})<1$ for any $1\leq i\leq N-1$, to obtain

[TABLE]

Since $\mu_{1}\geq\mu_{2}\geq\dots\geq\mu_{N-1}$, then

[TABLE]

So $(\ref{schur1stbound})$ implies

[TABLE]

Similar bounds can be achieved for types $C,D$. Recall that the probabilities we are dealing with are $M_{N}(\cdot|\lambda)$ after conditioning over values for $\nu_{1},\ldots,\nu_{N},\mu_{1},\ldots,\mu_{N-1}$. But the bound achieved did not take these values into consideration, therefore $\textrm{Prob}(\mu_{N}=b)/\textrm{Prob}(\mu_{N}=b+1)<q^{N}/(q;q)^{2}_{\infty}$, if $\mu_{N}$ is $M_{N}(\cdot|\lambda)$-distributed. The bound just shown holds for any $0\leq b\leq\lambda_{N+1}$. This implies the existence of a desired constant $c_{1}>0$ for $(\ref{firstline})$.

Next, let us prove $(\ref{secondline})$; the proof is similar to that of $(\ref{firstline})$. We can assume $\lambda_{N}\geq\lambda_{N+1}+1$, since otherwise $\textrm{Prob}(\mu_{N}=\lambda_{N+1}+1)=0$. For each $\lambda_{N}-1\geq b\geq\lambda_{N+1}$, we want to show the ratio

[TABLE]

is of order $O(q^{N})$. Define the measure $M_{N}^{\prime}(\cdot|\lambda)$ on $\mathbb{GT}^{+}_{N+1}\times\mathbb{GT}^{+}_{N}$ via

[TABLE]

As before, Proposition 2.9 and Proposition 2.5, see also Remark 2.6, show that $M_{N}^{\prime}(\cdot|\lambda)$ is a probability measure whose pushforward under the projection $\mathbb{GT}^{+}_{N+1}\times\mathbb{GT}^{+}_{N}\rightarrow\mathbb{GT}^{+}_{N}$ is $\Lambda^{N+1}_{N}(\lambda,\cdot)$. As we did before, instead of estimating the ratio $(\ref{symplecticratio2})$ with respect to the probability measure $\Lambda^{N+1}_{N}(\lambda,\cdot)$, we estimate it when (in both the numerator and denominator) $\mu$ is distributed according to $M_{N}^{\prime}(\cdot|\lambda)$ after conditioning over fixed values for $\nu_{1},\nu_{2},\ldots,\nu_{N-1},\nu_{N+1}$ and $\mu_{1},\ldots,\mu_{N-1}$. For any $\lambda_{N+1}\leq d\leq e\leq\lambda_{N}$, with respect to this probability measure, we get

[TABLE]

We know that $\tau^{G}(q^{-(N+\epsilon)};\lambda,\nu,\mu)$ depends only on $\lambda_{N+1},\nu_{N+1}$ and $\mu_{N}$, so let us denote it $\tau^{G}_{N}(\lambda_{N+1},\nu_{N+1},\mu_{N})$. The factor $\tau^{G}_{N}(\lambda_{N+1},\nu_{N+1},d)$ appears in each term of the first line of $(\ref{eqn:prob})$, so it can be factored out. On the other hand, from the formulas in Proposition 2.9, we obtain that $\textrm{Prob}(\mu_{N}=d)$ is proportional (up to a nonzero factor that does not depend on $d$) to:

[TABLE]

Next we note that for any $\lambda_{N}-1\geq b\geq\lambda_{N+1}$, we have $\tau^{G}_{N}(\lambda_{N+1},\nu_{N+1},b)=\tau^{G}_{N}(\lambda_{N+1},\nu_{N+1},b+1)$, except in the case that $G=D$, $b=0$, in which case $\tau^{D}_{N}(\lambda_{N+1},\nu_{N+1},0)=1$, $\tau^{D}_{N}(\lambda_{N+1},\nu_{N+1},1)=2$. Therefore the ratio $\textrm{Prob}(\mu_{N}=b+1)/\textrm{Prob}(\mu_{N}=b)$ equals

[TABLE]

Just as we have done already a few times, we can upper bound all these three expressions by $cq^{N}/(q;q)_{\infty}^{2}$, for some constant $c>0$; the bound is uniform on the values $\nu_{1},\ldots,\nu_{N-1},\nu_{N+1}$, $\mu_{1},\ldots,\mu_{N-1}$ that we conditioned over. Therefore, if $\mu_{N}$ is $M_{N}^{\prime}(\cdot|\lambda)$-distributed, we also have that $\textrm{Prob}\left(\mu_{N}=b+1\right)/\textrm{Prob}\left(\mu_{N}=b\right)$ is upper bounded by $cq^{N}/(q;q)^{2}_{\infty}$, for any $\lambda_{N}-1\geq b\geq\lambda_{N+1}$. The bound $(\ref{secondline})$ follows.

The general case $m\geq 2$ of $(\ref{generalreduction})$ is very similar. It suffices to show the existence of constants $c_{1},c_{2}>0$ such that

[TABLE]

Denote $\lambda(N+1)$ by $\lambda$, and $\mu(N)$ by $\mu$. For $(\ref{generalfirstline})$, we can assume $\lambda_{N+2-m}>\lambda_{N+3-m}$, since otherwise $\textrm{Prob}(\mu_{N+1-m}\leq\lambda_{N+2-m}-1)=0$. Then the inequality can be deduced if we show that, for each $\lambda_{N+3-m}\leq b<\lambda_{N+2-m}$, the ratio

[TABLE]

is of order $O(q^{N})$. As before, it suffices to consider $\mu_{N}$ with law $M_{N}(\cdot|\lambda)$ and to show the last statement with respect to this probability measure. In fact, we can even condition over any values of $\nu_{1},\ldots,\nu_{N+1-m},\nu_{N+3-m},\ldots,\nu_{N+1}$ and $\mu_{1},\ldots,\mu_{N-m},\mu_{N+2-m},\ldots,\mu_{N}$ such that $\lambda_{i}\geq\nu_{i}\geq\lambda_{i+1}$, $\nu_{j}\geq\mu_{j}\geq\nu_{j+1}$, for relevant $i,j$. For any $\lambda_{N+3-m}\leq e\leq d\leq\lambda_{N+2-m}$, we use

[TABLE]

Then we can repeat the analysis above to prove the desired bound. The only difference is that now $\tau^{G}(q^{N+\epsilon};\lambda,\nu,\mu)$ does not depend on $\mu_{N+1-m}$ or on $\nu_{N+2-m}$ (it only depends on $\lambda_{N+1},\nu_{N+1},\mu_{N}$), so we could even use instead

[TABLE]

which makes the remaining steps of the proof of $(\ref{generalfirstline})$ even simpler. For the proof of $(\ref{generalsecondline})$, the same outline works, in particular, the factor $\tau^{G}(q^{N+\epsilon};\lambda,\nu,\mu)$ does not play a role in the proof. ∎

6.3. Law of Large Numbers and the Martin boundary

Theorem 6.4** (Law of Large Numbers).**

Let $k\in\mathbb{N}$ be arbitrary, let $\mathbf{y}\in\mathfrak{Y}$, and $\{M_{N}^{\mathbf{y}}\}_{N\geq 1}$ be the coherent system of Proposition 6.2. Let $\{\mu(N)\in\mathbb{GT}^{+}_{N}\}_{N\geq 1}$ be a sequence of random nonnegative signatures such that each $\mu(N)$ is $M_{N}^{\mathbf{y}}$-distributed. Then, for any $k\in\mathbb{N}$, the probability of the event

[TABLE]

tends to $1$, as $L$ goes to infinity.

Proof.

The theorem is a consequence of Proposition 6.3; the proof is very similar to that of Theorem 5.5, and therefore we leave the details to the reader. ∎

The following theorem is the analogue of Theorem 5.6; its proof is similar and we omit it.

Theorem 6.5**.**

The Martin boundary of the BC type $q$-Gelfand-Tsetlin graph is in bijection with the set $\mathfrak{Y}$, under the map $\mathbf{y}\mapsto\{M_{N}^{\mathbf{y}}\}_{N\geq 1}$ of Proposition 6.2.

6.4. Characterization of the minimal boundary

The following is the main theorem of this section; its proof is similar to that of Theorem 5.7 and we omit it.

Theorem 6.6**.**

The minimal boundary $\Omega_{q}^{G}$ of the BC type $q$-Gelfand-Tsetlin graph is equal to the Martin boundary $\Omega_{q}^{G,\textrm{Martin}}$.