Reduced word enumeration, complexity, and randomization

TL;DR

This paper investigates the complexity and algorithms for enumerating reduced words of permutations, revealing exponential growth in certain statistics and analyzing related combinatorial structures with implications in algebraic geometry.

Contribution

It provides a formal runtime analysis of the transition algorithm and establishes exponential growth results for the Edelman-Greene statistic, extending to Hecke words and Young tableaux enumeration.

Findings

Edelman-Greene statistic is typically exponentially large

Transition algorithm's runtime analysis shows exponential complexity

Enumeration of Hecke words and Young tableaux is also exponential

Abstract

A reduced word of a permutation $w$ is a minimal length expression of $w$ as a product of simple transpositions. We examine the computational complexity, formulas and (randomized) algorithms for their enumeration. In particular, we prove that the Edelman-Greene statistic, defined by S. Billey-B. Pawlowski, is typically exponentially large. This implies a result of B. Pawlowski, that it has exponentially growing expectation. Our result is established by a formal run-time analysis of A. Lascoux-M.-P. Sch\"utzenberger's transition algorithm. The more general problem of Hecke word enumeration, and its closely related question of counting set-valued standard Young tableaux, is also investigated. The latter enumeration problem is further motivated by work on Brill-Noether varieties due to M. Chan-N. Pflueger and D. Anderson-L. Chen-N. Tarasca.

Equations

$$\#\mathsf{Red}(w)=\text{the coefficient of x1x2⋯xℓ(w) in Fw.}$$

$$\#\mathsf{Red}(w)=\sum _{\lambda }{a}_{w,\lambda }{f}^{\lambda },\text{ where}$$

$$f^{\lambda }=\frac{|\lambda |!}{{\prod }_b{h}_b},$$

$$\#\mathsf{Red}(w_0)=f^{(n-1,n-2,\dots ,3,2,1)};$$

$$\mathsf{EG}(w)=\sum _{\lambda }{a}_{w,\lambda };$$

$$\#\mathsf{Red}(w)=f^{\lambda (w)};$$

$$\#\mathsf{Red}(w)=\sum _{v\in \mathcal{L}(w)}{f}^{\lambda (v)};$$

$$\#\mathsf{Red}({\theta }_n)=f^{(n,n)}=C_n:=\frac{1}{n+1}\left(\genfrac{}{}{0ex}{}{2n}{n}\right).$$

$$D(w)=\{(i,j):1\le i,j\le n,j<w(i),i<w^{-1}(j)\}.$$

$$\mathcal{E}ss(w)=\{(i,j)\in D(w):(i+1,j),(i,j+1)\notin D(w)\}.$$

$$\#\mathsf{Red}(54278316)$$

$$=f^{4,3,3,3,1,1}+f^{4,4,3,2,1,1}+f^{5,4,2,2,1,1}+f^{5,3,3,2,1,1}$$

$$=730158.$$

$${\mathfrak{S}}_w=x_r{\mathfrak{S}}_{w^{\prime }}+\sum _{w^{\prime \prime }}{\mathfrak{S}}_{w^{\prime \prime }},$$

$$F_w=\underset{N\to \infty }{\lim }{\mathfrak{S}}_{1^N\times w}\in \mathbb{Z}[[x_1,x_2,\dots ]].$$

$$F_w(x_1,x_2,\dots ,x_n,0,0,\dots )={\mathfrak{S}}_{1^n\times w}(x_1,x_2,\dots ,x_n,0,0,\dots ).$$

$${\mathfrak{S}}_{1^n\times w}=J(x_1,x_2,\dots ,x_{2n})+\sum _{v\in \mathcal{L}(w)}{\mathfrak{S}}_{1^n\times v},$$

$$F_w(x_1,\dots ,x_n)=\sum _{v\in \mathcal{L}(w)}{F}_v(x_1,\dots ,x_n).$$

$$F_v(x_1,\dots ,x_n)=s_{\lambda (v)}(x_1,\dots ,x_n);$$

$$F_w(x_1,\dots ,x_n)=\sum _{v\in \mathcal{L}(w)}{s}_{\lambda (v)}(x_1,\dots ,x_n).$$

$$F_w(x_1,\dots ,x_{\ell (w)})=\sum _{\lambda }{a}_{w,\lambda }{s}_{\lambda }(x_1,\dots ,x_{\ell (w)})$$

$$\mathbb{P}(\mathsf{EG}(w)\ge 2^{\alpha n})\ge 1-\frac{1}{n^{2\gamma }}.$$

$$N_{2143,n}(u^{\prime })\ge N_{2143,n}(u).$$

$$A=(c,d)$$

$$B=(x^{\prime },y)$$

$$C=(x,y^{\prime })$$

$$\{i_1,i_2,i_3,i_4\}\cap \{c,x,x^{\prime }\}=\varnothing$$

$${\mathcal{F}}_u^{\prime \prime }={\mathcal{F}}_u\setminus {\mathcal{F}}_u^{\prime }.$$

$$\psi :{\mathcal{F}}_u^{\prime \prime }\hookrightarrow {\mathcal{F}}_{u^{\prime }}^{\prime \prime }.$$

$$\psi ([{\bullet }_2,A,{\bullet }_4,{\bullet }_3]):=[{\bullet }_2,B^{\prime },{\bullet }_4,{\bullet }_3].$$

$$\psi ([{\bullet }_2,A,{\bullet }_4,C]):=[{\bullet }_2,B^{\prime },{\bullet }_4,C^{\prime }].$$

Full text

1. Introduction

1.1. Reduced word combinatorics

Let $S_{n}$ denote the symmetric group on $\{1,2,\ldots,n\}$. Each $w\in S_{n}$ can be expressed as a product of $\ell(w)$ simple transpositions $s_{i}=(i,i+1)$, where $\ell(w)$ is the number of inversions of $w$, i.e., pairs $i<j$ such that $w(i)>w(j)$. Such an expression $w=s_{i_{1}}s_{i_{2}}\cdots s_{i_{\ell(w)}}$ is a reduced word for $w$.

Let ${\sf Red}(w)$ be the set of reduced words for $w$. R. P. Stanley [35] defined a symmetric function $F_{w}$ such that

[TABLE]

In connection to ibid., P. Edelman-C. Greene [13, Section 8] proved that

[TABLE]

• •

$f^{\lambda}$ is the number of standard Young tableaux of shape $\lambda$, that is, row and column increasing bijective fillings of the Young diagram of $\lambda$ using $1,2,\ldots,|\lambda|$. The hook-length formula of J. S. Frame-G. de B. Robinson-R. M. Thrall [16] states

[TABLE]

where the product is over all boxes $b\in\lambda$ and $h_{b}$ is the hooklength of $b$, i.e., the number of boxes weakly right and strictly below $b$.

• •

$a_{w,\lambda}$ counts EG tableaux: row and column increasing fillings $T$ of $\lambda$ such that reading the entries $(i_{1},i_{2},\ldots,i_{|\lambda|})$ of $T$ along columns, top to bottom, and right to left, gives a reduced word $s_{i_{1}}\cdots s_{i_{|\lambda|}}$ for $w$ (cf. [10]).

Let $w_{0}=n\ n-1\ n-2\ \ldots\ 3\ 2\ 1$ be the unique longest length permutation of $S_{n}$ (hence $\ell(w_{0})={n\choose 2}$). R. P. Stanley [35] proved that, in this case, (2) is short:

[TABLE]

hence $\#{\sf Red}(w_{0})$ is computed by (3).

One measure of the brevity of (2) is the Edelman-Greene statistic on $S_{n}$,

[TABLE]

this was introduced by S. Billey-B. Pawlowski [5]. From (4), one sees ${\sf EG}(w_{0})=1$. Permutations $w$ such that ${\sf EG}(w)=1$ are vexillary. These permutations are characterized by $2143$-pattern avoidance: there are no indices $i_{1}<i_{2}<i_{3}<i_{4}$ such that $w(i_{1}),w(i_{2}),w(i_{3}),w(i_{4})$ are in the same relative order as $2143$. For instance, $w=\underline{5}\underline{4}27\underline{8}31\underline{6}$ is not vexillary; the underlined positions give a $2143$ pattern. Each such $w$ has shape $\lambda(w)$ (defined in Section 2.2). Extending (4), whenever $w$ is vexillary,

[TABLE]

see, e.g., [29, Corollary 2.8.2]. Our main result (Theorem 3.2) is that EG is typically large. This implies a (weak version) of a Theorem of B. Pawlowski [31, Theorem 3.2.7]:

Theorem 1.1** (Average exponential growth).**

${\mathbb{E}}[{\sf EG}]=\Omega(c^{n})$, for some fixed constant $c>1$.

1.2. Computational complexity and transition

Our proof of Theorem 1.1 applies the transition algorithm of A. Lascoux-M. P. Schützenberger [26] (cf. [29, Sections 2.7, 2.8]). This algorithm constructs a tree ${\mathcal{T}}(w)$ whose root is $w$ and the leaves ${\mathcal{L}}(w)$ are labelled with vexillary permutations (with multiplicity). With this,

[TABLE]

see Section 2 for details. Different $v$ may give the same $\lambda(v)$. After combining such terms, (6) is the same as (2); see Lemma 3.1.

The (practical) efficiency of (extensions/variations of) transition has been mentioned a number of times. S. Billey [3] calls transition “one of the most efficient methods” to compute Schubert polynomials. See also A. Buch [9, Section 3.4] and Z. Hamaker-E. Marburg-B. Pawlowski [19]. On the other hand, concerning the application of transition to computing the Littlewood-Richardson coefficients [26], A. Garsia [18, p. 52] writes:

“Curiously, their algorithm (in spite of their claims to the contrary) is hopelessly inefficient as compared with well known methods.”

He also refers to transition as “efficient” for a different purpose in his study of ${\sf Red}(w)$.

Theorem 1.1 is actually a reformulation of the following result which is a formal complexity analysis of transition:

Theorem 1.2**.**

${\mathbb{E}}(\#\mathcal{L})=\Omega(c^{n})$* for a fixed constant $c>1$. That is the average running time of transition, as an algorithm to compute $\#{\sf Red}(w)$, is at least exponential in $n$.*

Theorem 3.2 strengthens Theorem 1.2 to show that the “typical” running time is exponentially large. To prove Theorem 1.2 we use that the expected number of occurences of a fixed pattern $\pi\in{S}_{k}$ in $w\in{S}_{n}$ is ${n\choose k}/k!$. Thus for $u=2143$, this expectation is $O(n^{4})$. One shows each step of transition reduces the number of $2143$ patterns by $O(n^{3})$. Using the graphical description of transition by A. Knutson and the third author [25], a node $u$ of ${\mathcal{T}}(w)$ has exactly one child $u^{\prime}$ only if $u^{\prime}$ has weakly more $2143$ patterns than $u$ does. Consequently, ${\mathcal{T}}(w)$ has $\Omega(n)$ branch points along any root-to-leaf path and thus exponentially many leaves. (In fact, the $c>1$ from our argument is close to $1$.)

Of course, the exponential average run-time of transition does not imply computing $\#{\sf Red}(w)$ is hard. Suppose one encodes a permutation $w$ by its Lehmer code ${\sf code}(w)=(c_{1},c_{2},\ldots,c_{L})$. What is the worst case complexity of computing $\#{\sf Red}(w)$ given input ${\sf code}(w)$?

L. Valiant [37] introduced the complexity class $\#{\sf P}$ of problems that count the number of accepting paths of a non-deterministic Turing machine running in polynomial time in the length of the input. Let ${\sf FP}$ be the class of function problems solvable in polynomial time on a deterministic Turing machine. It is basic theory that ${\sf FP}\subseteq\#{\sf P}$.

Observation 1.3**.**

$\#{\sf Red}(w)\not\in\#{\sf P}$. In particular, $\#{\sf Red}(w)\not\in{\sf FP}$.

Proof.

Let $\theta_{n}$ be the vexillary permutation with ${\sf code}(\theta_{n})=(n,n)$. Then, using (5),

[TABLE]

The middle equality is textbook: there is a bijection between standard Young tableaux of shape $(n,n)$ and Dyck paths from $(0,0)$ to $(2n,0)$; both are enumerated by the Catalan number $C_{n}$. Now, $\#{\sf Red}(\theta_{n})$ is doubly exponential in the input length $O(\log n)$. No such problem can be in $\#{\sf P}$ [30, Section 3]. ($\#{\sf Red}(w)\not\in{\sf FP}$ is true from this argument for the simple reason that it takes exponential time just to write down the output.) ∎

A counting problem ${\mathcal{P}}$ is $\#{\sf P}$-hard if any problem in $\#{\sf P}$ has a polynomial-time counting reduction to ${\mathcal{P}}$. Is $\#{\sf Red}(w)\in\#{\sf P}$-hard?

Observation 1.3 is dependent on the choice of encoding. For example, if one encodes a permutation $w\in S_{n}$ in the inefficient one-line notation, the input takes $O(n\log n)$ space. Since $\ell(w)\leq{n\choose 2}$ is polynomial in the input length, it follows that $\#{\sf Red}(w)\in\#{\sf P}$; see [34].

Problem 1.4**.**

Does there exist an $n^{O(1)}$-algorithm to compute $\#{\sf Red}(w)$?

It is easy to see that $\#{\sf Red}(u)\leq\#{\sf Red}(us_{i})$ whenever $\ell(us_{i})=\ell(u)+1$. Hence, $\#{\sf Red}(w)$ is maximized at $w=w_{0}$. So, by (4), $\log(\#{\sf Red}(w))\in n^{O(1)}$. Thus, unlike Observation 1.3, there is no easy negative solution to Problem 1.4 (and any negative solution implies ${\sf FP}\neq\#{\sf P}$, which is a famous open problem). Indeed, in the vexillary case (5), the hook-length formula (3) gives a $n^{O(1)}$-algorithm for $\#{\sf Red}(w)$.

1.3. Hecke words

Section 4 studies the more general problem of counting ${\sf Hecke}(w,N)$, the set of Hecke words of length $N$ whose Demazure product is a given $w\in{S}_{n}$. Here, the role of Stanley’s symmetric polynomial is played by the stable Grothendieck polynomial defined by S. Fomin and A. N. Kirillov [15]. Using work of S. Fomin and C. Greene [14] and of A. Buch, A. Kresch, M. Shimozono, H. Tamvakis and the third author [10], one has two analogues of the results of Edelman-Greene [13]. However, useful enumeration formulas for Hecke words, even when $w$ is vexillary, is a challenge.

As explained by Proposition 4.3, enumerating Hecke words is closely related to the problem of counting $f^{\lambda,N}$, the number of set-valued tableaux [8] that are $N$-standard of shape $\lambda$. These are fillings $T$ of the boxes of $\lambda$ by $1,2,\ldots,N$, where each entry appears exactly once, and if one chooses precisely one entry from each box of $T$, one obtains a semistandard tableau. For example, if $N=8$ and $\lambda=(3,2)$, one tableau is  

1,2

4,5

$8$

$3$

6,7

.

By Observation 1.3’s reasoning, (7) shows there is no algorithm to compute $f^{\lambda,N}$ that is polynomial-time in the bit-length of the input $(\lambda,N)$.

Problem 1.5**.**

Does there exist an algorithm to compute $f^{\lambda,N}$ that is polynomial in $|\lambda|$ and $N$?

Clearly, (3) gives a solution when $N=|\lambda|$. Using a theorem of C. Lenart [27], there exists an $|\lambda|^{O(1)}$ algorithm for any $\lambda$ and where $N=|\lambda|+k$, if $k$ is fixed (Proposition 4.5).

Recent work of M. Chan-N. Pflueger [11] and D. Anderson-L. Chen-N. Tarasca [2] motivates study of $f^{\lambda,N}$ in terms of Brill-Noether varieties. We remark on two manifestly nonnegative formulas for the Euler characteristics of these varieties (Corollary 4.10).

1.4. Randomization

Section 5 gives three randomized algorithms to estimate $\#{\sf Red}(w)$ and/or $\#{\sf Hecke}(w,N)$ using importance sampling. That is, let $S$ be a finite set. Assign $s\in S$ probability $p_{s}$. Let ${\sf Z}$ be a random variable on $S$ with $Z(s)=1/p_{s}$. Then ${\mathbb{E}}({\sf Z})=\sum_{s\in S}p_{s}\times\frac{1}{p_{s}}=\#S$. Using this, one can devise simple Monte Carlo algorithms to estimate $\#S$. The idea goes back to at least a 1951 article of H. Kahn-T. E. Harris [21], who furthermore credit J. von Neumann. The application to combinatorial enumeration was popularized through D. Knuth’s article [22] which applies it to estimating the number of self-avoiding walks in a grid. An application to approximating the permanent was given by L. E. Rasmussen [32]. More recently, J. Blitzstein-P. Diaconis [6] develop an importance sampling algorithm to estimate the number of graphs with a given degree sequence. We are suggesting another avenue of applicability, to core objects of algebraic combinatorics.

2. The Graphical Transition algorithm

2.1. Preliminaries

The graph $G(w)$ of a permutation $w\in{S}_{n}$ is the $n\times n$ grid, with a $\bullet$ placed in position $(i,w(i))$ (in matrix coordinates). The Rothe diagram of $w$ is given by

[TABLE]

Pictorially, this is described by striking out boxes below and to the right of each $\bullet$ in $G(w)$. $D(w)$ consists of the remaining boxes. If it exists, the connected component involving $(1,1)$ is the dominant component. The essential set of $w$ consists of the maximally southeast boxes of each connected component of $D(w)$, i.e.,

[TABLE]

If it exists, the accessible box is the southmost then eastmost essential set box not in the dominant component. For example, if $w=54278316\in{S}_{8}$, $D(w)$ is depicted by:

[TABLE]

Also, ${\mathcal{E}ss}(w)=\{(1,4),(2,3),(5,3),(5,6),(6,1)\}$, and the accessible box is at $(5,6)$. The Lehmer code of $w\in S_{\infty}$, denoted ${\sf code}(w)$ is the vector $(c_{1},c_{2},\ldots,c_{L})$ where $c_{i}$ equals the number of boxes in row $i$ of the Rothe diagram of $w$. We will assume $L$ is minimum (i.e., ${\sf code}(w)$ does not have trailing zeros). By this convention, ${\sf code}(id)=()$.

Fulton’s criterion [17, Remark 9.17] states that $u$ is vexillary if and only if there does not exist two essential set boxes where one is strictly northwest of the other. Thus, using the above picture of $D(w)$ we can see that $w$ is not vexillary because of, e.g., $(1,4)$ and $(5,6)$.

2.2. Description of ${\mathcal{T}}(w)$

The original description of transition was given in [26]; this account is also given an exposition in [29, Sections 2.7.3, 2.7.4, 2.8.1]. We follow the graphical description given in [25] and its elaboration in [1]. There are some minor choices in describing the transition tree, and those of [25, 1] differ slightly from [26, 29].

We describe the graphical version of the transition algorithm to compute $\#{\sf Red}(w)$. The root of the tree is labelled by $D(w)$. If $w$ is vexillary, stop. Otherwise, there exists an accessible box. (If not, $D(w)$ consists only of the dominant component and, by Fulton’s criterion, $w$ is vexillary, a contradiction.) The pivots of $D(w)$ are the maximally southeast $\bullet$’s of $G(w)$, say $b_{1},b_{2},\ldots,b_{t}$ that are northwest of the accessible box $e$.

If $w$ is not vexillary, the children of $w$ are defined as follows. For each $i=1,2,\ldots,t$, let $R_{i}$ be the rectangle defined by $b_{i}$ and $e$. Remove $b_{i}$ and its rays from $G(w)$ to form $G^{(i)}(w)$. Order the boxes $\{v_{i}\}_{i=1}^{m}$ in English reading order. Move $v_{1}$ strictly north and strictly west to the closest position not occupied by another box of $D(w)$ or a ray from $G^{(i)}(w)$. Now, iterate this procedure with $v_{2},v_{3},\ldots$. At each step, $v_{j}$ may move to a position vacated by earlier moves. The result is the diagram $D(w^{(i)})$ of some permutation $w^{(i)}$. These $D(w^{(i)})$’s are the children of $D(w)$. We call the transformation $D(w)\to D(w^{(i)})$ a marching move.

Example 2.1*.*

Continuing our example, the pivots of $w$ are $(1,5),(2,4)$ and $(3,2)$. We now obtain the child corresponding to the pivot $b_{2}=(2,4)$:

$w$remove hook at $(2,4)$XX$w^{(2)}=56274318$

We have indicated by “X” the boxes that have moved. This process constructs one of the three children of $w$. In Figure 1 we draw the remainder of ${\mathcal{T}}(w)$.∎

If $u$ is vexillary we define $\lambda(u)$ graphically by pushing all boxes of $D(u)$ northwest along the diagonal that it sits until a partition shape is reached; see [24, Section 3.2]. Concluding our running example, from Figure 1 we have

[TABLE]

This result is a mild variation of [29, Proposition 2.8.1] (cf. [26]) using the marching moves. We make no claim of originality.

Theorem 2.2** (cf. [26, 29, 25]).**

$\#{\sf Red}(w)=\sum_{v\in{\mathcal{L}}(w)}f^{\lambda(v)}$.

Proof: We follow [1, Section 5.2], which elaborates on the notions from [25] in the case of Schubert polynomials ${\mathfrak{S}}_{w}$. We refer to [29, Chapter 2] for background.

Let $(r,c)$ be the accessible box of $w\in S_{n}$ and set $k=w^{-1}(c)$. Also let $w^{\prime}=w\cdot(r,k)$. Transition gives this recurrence for the Schubert polynomials:

[TABLE]

where the summation is over the children $w^{\prime\prime}$ of $w$ in ${\mathcal{T}}(w)$.

Let $1^{N}\times w\in S_{N+n}$ send $j\mapsto j$ for $1\leq j\leq n$ and $j\mapsto w(j-N+1)+N$ for $j\geq N$. Then

[TABLE]

Moreover, since $w\in{S}_{n}$ then

[TABLE]

Now, by repeated application of (8) to $1^{n}\times w$,

[TABLE]

where $J(x_{1},x_{2},\ldots,x_{n},0,0,\ldots)\equiv 0$.

Hence by setting $x_{i}=0$ for $i>n$ in (10) we obtain, using (9) that

[TABLE]

Let $s_{\alpha}(x_{1},\ldots,x_{n})$ be the Schur polynomial for a shape $\alpha$. Since $v\in{\mathcal{L}}(w)$ is vexillary,

[TABLE]

see, e.g., [29, Section 2.8.1]. Hence

[TABLE]

We have that $[x_{1}x_{2}\cdots x_{\ell(w)}]F_{w}=\#{\sf Red}(w)$ and $[x_{1}x_{2}\cdots x_{\ell(w)}]s_{\lambda(v)}(x_{1},\ldots,x_{n})=f^{\lambda(v)}$. Now the result follows from these two facts combined with (12).∎

3. Proof of Theorems 1.1 and 1.2

3.1. On the distribution of ${\sf EG}(w)$

Lemma 3.1**.**

For any $w\in{S}_{n}$, ${\sf EG}(w)=\#{\mathcal{L}}(w)$.

Proof.

Combining results of [35, 13] gives

[TABLE]

where the sum is over partitions $\lambda$ of size $\ell(w)$, and $a_{w,\lambda}$ is defined in Section 1.

The Schur polynomials $s_{\lambda}(x_{1},\ldots,x_{\ell(w)})$ for $|\lambda|=\ell(w)$ are a basis of the vector space $\Lambda^{(\ell(w))}_{\mathbb{Q}}[x_{1},\ldots,x_{\ell(w)}]$ of degree $\ell(w)$ symmetric polynomials in $\{x_{1},\ldots,x_{\ell(w)}\}$. Since (13) and (12) (where $n=\ell(w)$) are linear combinations for the same vector, we are done. ∎

In view of Lemma 3.1, Theorems 1.1 and 1.2 are equivalent. It is easy to see that Theorem 1.1 follows from our main result:

Theorem 3.2**.**

Fix $0<\gamma<\frac{1}{2}$. There exists $\alpha>0$ such that for $n$ sufficiently large,

[TABLE]

Proof of Theorem 3.2: Let $N_{\pi,n}(w)$ be the number of $\pi$ patterns contained in $w\in S_{n}$.

Proposition 3.3**.**

Suppose in ${\mathcal{T}}(w)$ that the node $u$ has exactly one child $u^{\prime}$. then

[TABLE]

Proof of Proposition 3.3: Let the accessible box ${\bf z}_{u}$ of $u$ be in position $(x,y)$. By definition of $D(u)$, there is a $\bullet$ of $G(u)$ at $C=(x,y^{\prime})$ for some $y^{\prime}>y$, and there is a $\bullet$ at $B=(x^{\prime},y)$ for some $x^{\prime}>x$. Let $b_{1}$ be the unique pivot of $D(u)$, i.e., the southeastmost $\bullet$ that is northwest of ${\bf z}_{u}$ (as in Section 2.2). Suppose $b_{1}$ is at position $A=(c,d)$. Thus, $c<x$ and $d<y$.

By definition of the transition algorithm, all $\bullet$’s of $G(u)$ and $G(u^{\prime})$ are in the same position, except $A,B,C$ in $G(u)$ are respectively replaced by $A^{\prime},B^{\prime},C^{\prime}$ in $G(u^{\prime})$ where

[TABLE]

Schematically, the march move looks as follows (we have thickened the moving $\bullet$’s).

[TABLE]

Claim 3.4**.**

If there are two $\bullet$’s, other than $\{B,C\}$, that are weakly south and weakly east of ${\bf z}_{u}$ then one $\bullet$ must be (strictly) southeast of the other.

Proof of Claim 3.4: Suppose not. Then let the two $\bullet$’s be at $(q,r)$ and $(m,n)$ where $q>m$ and $r<n$. Then there is a box ${\bf z}\neq{\bf z}_{u}$ of $D(u)$ in position $(m,q)$, which is weakly south and weakly east of ${\bf z}_{u}$. Since ${\bf z}_{u}$ is not in the dominant component of $D(u)$, then ${\bf z}$ cannot be in that component either. Therefore, ${\bf z}_{u}$ is not the accessible box of $D(u)$, a contradiction. ∎

Claim 3.5**.**

There is no $\bullet$ of $G(u)$ strictly north of row $c$ and strictly between columns $d$ and $y$. Similarly, there is no $\bullet$ of $G(u)$ strictly west of column $d$ and strictly between rows $c$ and $x$.

Proof of Claim 3.5: We prove only the first sentence of the claim, as the second sentence is analogous. Suppose not; we may assume this $\bullet$ is maximally southeast with the assumed properties. Then $A=(c,d)$ and this $\bullet$ are two pivots for $D(u)$, which implies $u$ has at least two children, contradicting the hypothesis of the Proposition. ∎

Let ${\mathcal{F}}_{u}$ consist of all embedding positions $i_{1}<i_{2}<i_{3}<i_{4}$ of a $2143$-pattern in $u$. Also, let ${\mathcal{F}}^{\prime}_{u}$ be the subset of ${\mathcal{F}}_{u}$ consisting of those $i_{1}<i_{2}<i_{3}<i_{4}$ such that

[TABLE]

(i.e., the positions do not involve the rows of $A,B$ or $C$). Let

[TABLE]

Similarly, we define ${\mathcal{F}}_{u^{\prime}}$, ${\mathcal{F}}^{\prime}_{u^{\prime}}$ and ${\mathcal{F}}^{\prime\prime}_{u^{\prime}}$ in exactly the same way, except with respect to $u^{\prime}$.

Since ${\mathcal{F}}^{\prime}_{u}={\mathcal{F}}^{\prime}_{u^{\prime}}$, it suffices to establish an injection

[TABLE]

In what follows, we will let $\bullet_{i}$ refer to the $\bullet$ in the diagram corresponding to the “$i$” in the $2143$ pattern, for $1\leq i\leq 4$. In addition, if $i_{1}$ is in the row of $A$ we will write “$A=\bullet_{2}$”, etc. We define now $\psi$ in cases:

Case 1: ($B=\bullet_{1}$ or $B=\bullet_{2}$) The $\bullet_{4}$ and $\bullet_{3}$ appear strictly right of column $y$. This contradicts Claim 3.4. Hence, no elements of ${\mathcal{F}}^{\prime\prime}_{u}$ fall into this case.

Case 2: ($C=\bullet_{1}$ or $C=\bullet_{2}$) $\bullet_{4}$ and $\bullet_{3}$ appear strictly southeast of ${\bf z}_{u}$. As in Case 1, this contradicts Claim 3.4. Again, no elements of ${\mathcal{F}}^{\prime\prime}_{u}$ fall into this case.

Case 3: ($A=\bullet_{1}$) Let $\bullet_{2}$ be at position $(r,s)$. Hence $r<c$ and $s>d$. If moreover, $s<y$ we contradict the first sentence of Claim 3.5. Hence, $s>y$. We must have that $\bullet_{2}\not\in\{A,B,C\}$ and $\bullet_{4}$ and $\bullet_{3}$ are strictly to the right of column $y$.

Subcase 3a: ($\bullet_{4}$ and $\bullet_{3}$ are both strictly south of row $x$) This contradicts Claim 3.4.

Subcase 3b: ($\bullet_{4}$ and $\bullet_{3}$ are both strictly north of row $x$) Thus $\{\bullet_{3},\bullet_{4}\}\cap\{A,B,C\}=\emptyset$. The $2143$ pattern $[\bullet_{2},A,\bullet_{4},\bullet_{3}]$ is destroyed by the marching move, i.e., $[\bullet_{2},A^{\prime},\bullet_{4},\bullet_{3}]$ is not a $2143$ pattern in $u^{\prime}$. Now, in $u^{\prime}$ we now have the $2143$ pattern $[\bullet_{2},B^{\prime},\bullet_{4},\bullet_{3}]$. Hence we define

[TABLE]

Subcase 3c: ($\bullet_{4}$ is strictly north of row $x$ and $\bullet_{3}$ is strictly south of row $x$). Since $s>y$, $C\neq\bullet_{3}$. Hence $\{\bullet_{3},\bullet_{4}\}\cap\{A,B,C\}=\emptyset$. The $2143$ pattern $[\bullet_{2},A,\bullet_{4},\bullet_{3}]$ is destroyed by the marching move. However, in $u^{\prime}$ we now have the $2143$ pattern $[\bullet_{2},B^{\prime},\bullet_{4},\bullet_{3}]$. We again define

[TABLE]

Subcase 3d: ($\bullet_{3}$ is in row $x$ and $\bullet_{4}$ is strictly above row $x$) Then in fact $\bullet_{3}=C$ while $\bullet_{4}\not\in\{A,B,C\}$. In this case,

[TABLE]

Subcase 3e: ($\bullet_{4}$ is in row $x$ and $\bullet_{3}$ is strictly south of row $x$) Thus $\bullet_{4}=C$ and $\bullet_{3}$ is strictly southeast of $z_{u}$. This contradicts Claim 3.4.

Case 4: ($A=\bullet_{2}$) Let the $1$ be at position $(r,s)$. Hence $r>c$ and $s<d$. If $r\leq x$ then we contradict the second sentence of Claim 3.5. Hence $r>x$. We have that $\bullet_{4}$ and $\bullet_{3}$ are in rows strictly south of $x$. Moreover, there must be a box $e$ of $D(u)$ in the row of $\bullet_{4}$ and the column of $\bullet_{3}$ that is therefore strictly south of ${\bf z}_{u}$. Since the columns of $\bullet_{4}$ and $\bullet_{3}$ are strictly east of column $d$, the box $e$ is not part of the dominant component of $D(u)$. Hence, ${\bf z}_{u}$ cannot be the accessible box, a contradiction. Thus, no elements of ${\mathcal{F}}^{\prime\prime}_{u}$ are in this case.

Case 5: ($A=\bullet_{3}$) Hence, in $u$, $\bullet_{2},\bullet_{1},\bullet_{4}$ are strictly north of row $c$. Thus $\{\bullet_{1},\bullet_{2},\bullet_{4}\}\cap\{A,B,C\}=\emptyset$ and $\bullet_{2},\bullet_{1},\bullet_{4}$ remain in the same place in $u^{\prime}$. Set

[TABLE]

Case 6: ($A=\bullet_{4}$) $\bullet_{3}$ is strictly south of the row of $A$. If it is also weakly north of $x$, we contradict the second sentence of Claim 3.5. Hence $\bullet_{3}$ is strictly south of $x$, i.e., the row of $e$. Now, $\bullet_{2},\bullet_{1},\bullet_{3}$ are the same position in $u$ and $u^{\prime}$ and $\{\bullet_{1},\bullet_{2},\bullet_{3}\}\cap\{A,B,C\}=\emptyset$. Here,

[TABLE]

Since the row of $A^{\prime}$ is $x$ the output is a $2143$ pattern in $u^{\prime}$.

Case 7: ($B=\bullet_{4}$) Let $\bullet_{3}$ be at $(r,s)$. Thus $r>x^{\prime}$ and $s<y$. There must be a box $e\in D(w)$ in position $(x^{\prime},s)$. Now, $\bullet_{2}$ and $\bullet_{1}$ are in columns strictly left of $s$ and strictly above row $r$. Hence $e$ cannot be in the dominant component of $D(w)$. Thus, since $e$ is further south than ${\bf z}_{u}$, the latter is not accessible, a contradiction. So, no elements of ${\mathcal{F}}^{\prime\prime}_{u}$ appear in this case.

Case 8: ($B=\bullet_{3}$) Let $\bullet_{4}$ be in position $(r,s)$.

Subcase 8a: ($r<c$) Therefore, $\bullet_{1}$ and $\bullet_{2}$ are also strictly above row $c$. Since $\bullet_{1},\bullet_{2}$ and $\bullet_{4}$ stay in the same place in $u$ and $u^{\prime}$ and $B^{\prime}$ is in row $c$ in $u^{\prime}$. Moreover, $\{\bullet_{1},\bullet_{2},\bullet_{4}\}\cap\{A,B,C\}=\emptyset$. We may define

[TABLE]

Subcase 8b: ($x<r<x^{\prime}$) This contradicts Claim 3.4.

Subcase 8c: $(r=c)$ This implies $A=\bullet_{4}$, which is impossible.

Subcase 8d: ($c<r<x$) We may assume $A\neq\bullet_{1},\bullet_{2}$ since those cases are handled by Case 3 and Case 4. Now, $\bullet_{1}$ and $\bullet_{2}$ are strictly west of column $y$ and strictly north of row $x$. By the assumption that $A=b_{1}$ is the (unique) pivot, combined with Claim 3.5, both $\bullet_{1}$ and $\bullet_{2}$ are strictly northwest of $A$. Thus, $\bullet_{1}$ and $\bullet_{2}$ are in the same place in $u^{\prime}$, and $\{\bullet_{1},\bullet_{2},\bullet_{4}\}\cap\{A,B,C\}=\emptyset$. Since $A^{\prime}$ is in row $x$, it make sense to let

[TABLE]

Subcase 8e: $(r=x)$ Hence $C=\bullet_{4}$. For the same reasons as in Subcase 8d, both $\bullet_{1}$ and $\bullet_{2}$ are strictly northwest of $A$. Thus, $\bullet_{1}$ and $\bullet_{2}$ are in the same place in $u^{\prime}$ and $\{\bullet_{1},\bullet_{2}\}\cap\{A,B,C\}=\emptyset$. In this case set

[TABLE]

Case 9: ($C=\bullet_{3}$) Let $\bullet_{4}$ be in position $(r,s)$. Hence $s>y^{\prime}$.

Subcase 9a: ($r<c$) Hence $\bullet_{1},\bullet_{2}$ and $\bullet_{4}$ remain in the same place in $u^{\prime}$ and $\{\bullet_{1},\bullet_{2},\bullet_{4}\}\cap\{A,B,C\}=\emptyset$. Since $C^{\prime}$ is further south than $C$, we may set

[TABLE]

Subcase 9b: ($c<r<x$) We may also assume that $A\neq\bullet_{1}$ and $A\neq\bullet_{2}$, since those are handled in Case 3 and Case 4, respectively. Thus $\{\bullet_{1},\bullet_{2},\bullet_{4}\}\cap\{A,B,C\}=\emptyset$. Here,

[TABLE]

Subcase 9c: $(r=c)$ Then $A=\bullet_{4}$, which is impossible.

Case 10: ($C=\bullet_{4}$) We may assume that $A\neq\bullet_{1},\bullet_{2}$ (Case 3 and Case 4) and also $B\neq\bullet_{3}$ (Case 8). Therefore $\{\bullet_{1},\bullet_{2},\bullet_{3}\}\cap\{A,B,C\}=\emptyset$. Let $\bullet_{3}$ be in position $(r,s)$.

Subcase 10a: ($y<s<y^{\prime}$) This contradicts Claim 3.4.

Subcase 10b: ($s=y$) This means $\bullet_{3}=B$, a situation we have ruled out/refer to Case 8.

Subcase 10c: ($s<y$) If moreover $r>x^{\prime}$ then there exists $e\in D(w)$ in position $(x^{\prime},s)$, which is therefore strictly south of ${\bf z}_{u}$. Since column $s$ is strictly east of the column of $\bullet_{2}$, $e$ is not in the dominant component. Hence ${\bf z}_{u}$ is not accessible, a contradiction. Now $r\neq x^{\prime}$ (since we assumed $B\neq\bullet_{3}$). Thus, $x<r<x^{\prime}$ and it follows that $\bullet_{3}$ is in the same place in $u^{\prime}$. By the reasoning of the first paragraph of Subcase 8d, $\bullet_{1},\bullet_{2}$ are strictly northwest of $A$. Hence $\bullet_{1},\bullet_{2}$ also remain in the same place in $u^{\prime}$. Summing up, since $B^{\prime}$ is in row $c$, we may define

[TABLE]

$\psi$ is well-defined: The above cases handle each of the possibilities for $A,B,C$ being one of $1,2,3,4$. Our definition of $\psi$ is shown to send an element of ${\mathcal{F}}^{\prime\prime}_{u}$ to an element of ${\mathcal{F}}^{\prime\prime}_{u^{\prime}}$.

We also need that if an element of ${\mathcal{F}}^{\prime\prime}_{u}$ occurs in two cases, $\psi$ sends them to the same element of ${\mathcal{F}}^{\prime\prime}_{u^{\prime}}$. By inspection, the only overlapping situations are Subcase 3d$\leftrightarrow$Subcase 9b and Subcase 8d$\leftrightarrow$Case 10. In both these cases we define $\psi$ to be consistent on the overlap.

$\psi$ is an injection: This is by inspection of pairs of subcases where $\psi$’s output was given. By our choice of notation, if $\bullet_{i}$ appears in the description of the input to $\psi$, it cannot be equal to $A,B$ or $C$ and hence in the output, it cannot be equal to $A^{\prime},B^{\prime}$ or $C^{\prime}$ (as $\{A,B,C\}$ and $\{A^{\prime},B^{\prime},C^{\prime}\}$ occupy the same rows). Therefore, if in two cases, some coordinate of the two outputs differ symbolically, those outputs cannot be equal. After ruling out these pairs, we are left with a few to check:

Subcase 3b, Subcase 3c: These differ in the fourth coordinate since in the former case, $\bullet_{3}$ is strictly north of row $x$ and in the latter case, $\bullet_{3}$ is strictly south of row $x$.

Case 5 and Subcase 8d: These differ in the third coordinate since in the former case, $\bullet_{4}$ appears above row $c$ whereas in the latter case, $\bullet_{4}$ is below row $c$.

Subcase 9a and Subcase 9b: These differ in the third coordinate for the same reason as the previous pair.∎

Lemma 3.6**.**

Let $w\in{S}_{n}$ and suppose $u\to u^{\prime}$ in ${\mathcal{T}}(w)$. Then

[TABLE]

Proof of Lemma 3.6: Since $u\to u^{\prime}$ in ${\mathcal{T}}(w)$, exactly three positions $a,i,j$ differ between $u$ and $u^{\prime}$. We are claiming that

[TABLE]

Let $t_{1}<t_{2}<t_{3}<t_{4}$ be the indices of a $2143$-pattern in $u$. First suppose $\{t_{1},t_{2},t_{3},t_{4}\}\cap\{a,i,j\}=\emptyset$. Clearly, $t_{1}<t_{2}<t_{3}<t_{4}$ are indices of a $2143$-pattern in $u^{\prime}$. Therefore this case does not contribute to $N_{2143,n}(u)-N_{2143,n}(u^{\prime})$.

Next assume $\#(\{t_{1},t_{2},t_{3},t_{4}\}\cap\{a,i,j\})=1$. There are ${3\choose 1}$ choices for which of $a,i$ or $j$ is in $\{t_{1},t_{2},t_{3},t_{4}\}$. Then there are at most ${n\choose 3}$ choices for $\{t_{1},t_{2},t_{3},t_{4}\}\setminus\{a,i,j\}$. Finally there are $4$ choices for which $k$ satisfies $t_{k}\in\{a,i,j\}$. Therefore, this case contributes at most ${3\choose 1}4{n\choose 3}$ to $N_{2143,n}(u)-N_{2143,n}(u^{\prime})$, thus explaining the third term of (14).

Similar arguments explain the first and second terms of (14) as the contributions to $N_{2143,n}(u)-N_{2143,n}(u^{\prime})$ from the cases that

[TABLE]

respectively. The lemma thus follows. ∎

The following is known; see work of M. Bona [7] and of S. Janson, B. Nakamura, and D. Zeilberger [20]. The proof being not difficult, we include it for completeness.

Lemma 3.7**.**

For any $\pi\in S_{k}$, the expected number of occurrences of $\pi$ as a pattern in $w\in{S}_{n}$ (selected using the uniform distribution) is ${n\choose k}/k!$.

Proof of Lemma 3.7: For an increasing sequence $I=\{i_{1}<i_{2}<\ldots<i_{k}\}$ (in $[1,n]$), let

[TABLE]

Thus, $N_{\pi,n}=\sum_{I}{\sf X}_{I}$. There are ${n\choose k}(n-k)!$ permutations such that $I$ has pattern $\pi$. By linearity of expectation,

[TABLE]

and the lemma follows.∎

Lemma 3.8**.**

Let ${\mathcal{T}}$ be a rooted tree with the property that along any path from the root to a leaf there are $d$ nodes with at least two children. Then that tree has at least $2^{d}$ leaves.

Proof of Lemma 3.8: Arbitrarily left-right order the descendants of the root of ${\mathcal{T}}$. After pruning, if necessary, we may assume each node as at most two children. Along any path from the root to a leaf, record “$S$” if a node has one child, and “$L$” if one steps to the left child and “$R$” if one goes to the right child. Thus, each leaf is uniquely encoded by an $\{S,L,R\}$ sequence. By hypothesis, each such sequence has at least $d$ from $\{L,R\}$. Also, each of the $2^{d}$-many $\{L,R\}$-sequences must be a subsequence of a unique leaf sequence. Hence there are at least $2^{d}$ leaves. ∎

By Chebyshev’s inequality, for any $t\in{\mathbb{R}}_{>0}$, ${\mathbb{P}}(|N_{\pi,n}-\mu|\geq t\sigma)\leq 1/t^{2}$, and hence

[TABLE]

For $\pi=2143$, $\mu={n\choose 4}/4!$. Let $t=n^{\gamma}$ for the fixed choice $0<\gamma<\frac{1}{2}$. Thus, we obtain

[TABLE]

Define a random variable $Q:S_{n}\to{\mathbb{Z}}_{\geq 0}$ by

[TABLE]

By Proposition 3.3 and Lemma 3.6,

[TABLE]

Combining (16) and (17) gives

[TABLE]

By [20, Section 4.1], the $r$-th central moment for $N_{\pi,n}$, i.e., ${\mathbb{E}}[(N_{\pi,n}-{\mathbb{E}}(N_{\pi,n}))^{r}]$, is a polynomial in $n$ of degree $\lfloor r(k-\frac{1}{2})\rfloor$ where, recall, $\pi\in S_{k}$. Hence ${\sf Var}(N_{2143,n})\in O(n^{7})$ and $\sigma\in O(n^{3.5})$. Therefore there exists $\alpha>0$ such that for $n$ sufficiently large

[TABLE]

Finally, by Lemma 3.1 and Lemma 3.8,

[TABLE]

The desired equality holds by (18) and (19) combined.∎

3.2. Remarks

M. Bona [7] proves that the sequence of random variables

[TABLE]

is asymptotically normal, i.e., $X_{n}$ converges in distribution to the standard normal variable $N(0,1)$. In particular, this means that for any $\epsilon>0$, for any $a,b\in{\mathbb{R}}$, there exists $N\in{\mathbb{N}}$ such that for all $n\geq N$, $|{\mathbb{P}}({\widetilde{X}}_{n}\in[a,b])-{\mathbb{P}}(N(0,1)\in[a,b])|<\epsilon$. Thus one could use Bona’s theorem to prove a more refined version of Theorem 3.2. However, this does not affect our basic conclusions, so we opted to state a result/proof that only appeals to Chebyshev’s inequality.

In [5], $w\in{S}_{n}$ is defined to be $k$-vexillary if ${\sf EG}(w)=k$. I. G. Macdonald [28] proves that the proportion of vexillary permutations in ${S}_{n}$ goes to zero as $n\to\infty$. Extending this, Theorem 3.2 implies:

Corollary 3.9**.**

Fix a positive integer $k$. Then \lim_{n\to\infty}{\mathbb{P}}(w\in{S}_{n}\text{\ is k-vexillary})\to 0.

Using the relations

[TABLE]

one can transform between any two reduced words

[TABLE]

see, e.g., [29, Proposition 2.1.6]. Hence, it follows that

[TABLE]

Let $\sigma^{(n)}=214365\cdots 2n\ 2n-1\in{S}_{2n}$.

Proposition 3.10**.**

$a_{\sigma^{(n)},\lambda}=f^{\lambda}$.

Proof.

Fix any partition $\lambda$ of size $2n-1$. Consider any row and column increasing filling $T$ of $\lambda$, using each of the labels $\{1,3,5,\ldots,2n-1\}$ precisely once. Let ${\mathcal{A}}_{\lambda}$ be the set of these tableaux. Also, let ${\mathcal{B}}_{\lambda}$ be the set of EG tableaux for the coefficient $a_{\sigma^{(n)},\lambda}$. ${\sf Red}(\sigma^{(n)})$ consists of all $n!$ rearrangements of the factors of $s_{1}s_{3}\cdots s_{2n-1}$. Hence, the column reading word of any $T\in{\mathcal{A}}_{\lambda}$ gives a reduced word for $w$. Thus, ${\mathcal{A}}_{\lambda}\subseteq{\mathcal{B}}_{\lambda}$. By (21), if $S\in{\mathcal{B}}_{\lambda}$, it must use each label of $\{1,3,5,\ldots,2n-1\}$ exactly once. Since $S$ must also be row and column increasing, we see $S\in{\mathcal{A}}_{\lambda}$. This gives ${\mathcal{A}}_{\lambda}={\mathcal{B}}_{\lambda}$.

Given $T\in{\mathcal{A}}_{\lambda}(={\mathcal{B}}_{\lambda})$, let $\phi(T)\in{\sf SYT}(\lambda)$ be the standard Young tableau of shape $\lambda$ obtained by sending label $i$ in $T$ to $\left\lceil\frac{i}{2}\right\rceil$. Clearly, $\phi:{\mathcal{A}}_{\lambda}\to{\sf SYT}(\lambda)$ is a bijection. Hence $a_{\sigma^{(n)},\lambda}=\#{\mathcal{A}}_{\lambda}=\#{\sf SYT}(\lambda)=f^{\lambda}$. ∎

Let ${\sf inv}(n)$ be the number of involutions of ${S}_{n}$. The following shows that the worst case and average case running time of transition is quite different:

Corollary 3.11**.**

$\#{\mathcal{L}}(\sigma^{(n)})={\sf inv}(n)\sim\left(\frac{n}{e}\right)^{n/2}\frac{e^{\sqrt{n}}}{(4e)^{\frac{1}{4}}}$.

Proof.

The equality holds since

[TABLE]

The first equality of (22) is Lemma 3.1, the second is Proposition 3.10 and the third is textbook (e.g., [36, Corollary 7.13.9]). The asymptotic statement is [23, Section 5.1.4]. ∎

The following conjecture has been proved by G. Orelowitz (private communication):

Conjecture 3.12**.**

$a_{w,\lambda}\leq f^{\lambda}$.

We refer to his paper (in preparation) for application to the Edelman-Greene statistic.

4. Counting Hecke words

A sequence $(i_{1},i_{2},\ldots,i_{N})$ is a Hecke word for $w\in{S}_{n}$ if $s_{i_{1}}\star s_{i_{2}}\star\cdots\star s_{i_{N}}=w$ where $\star$ is the Demazure product defined by

[TABLE]

Therefore, $N\geq\ell(w)$. Let ${\sf Hecke}(w,N)$ denote the set of Hecke words for $w$ of length $N$.

4.1. Two generalizations of the Edelman-Greene formula (2)

We now give two formulas for computing ${\sf Hecke}(w,N)$. Both are known to experts, but we are unaware of any specific place that they appear in the literature.

Since ${\sf Hecke}(w,\ell(w))={\sf Red}(w)$, formula (23) below generalizes (2). Our second point is that in contrast with (5), even for vexillary permutations, (23) is not short.

Proposition 4.1**.**

There is a manifestly nonnegative combinatorial formula

[TABLE]

where $b_{w,\lambda}$ counts the number of row strictly increasing and column weakly increasing tableaux of shape $\lambda$ whose top to bottom, right to left, column reading word is a Hecke word for $w$.

Let $M\geq 1$. There is a vexillary permutation $\pi\in S_{2M}$ with $\ell(\pi)=M^{2}$ such that

[TABLE]

where ${\sf par}(M)$ is the number of partitions of size $M$. That is when $w=\pi$ and $N=M^{2}+M$, (23) has at least ${\sf par}(M)$-many terms. Moreover,

[TABLE]

Proof.

We use the results of S. Fomin-A. N. Kirillov [15] who prove the following combinatorial formula for the stable Grothendieck polynomial $G_{w}$:

[TABLE]

where ${\bf i}=(i_{1},\ldots,i_{N})\in{\sf Hecke}(w,N)$, and ${\bf j}=(j_{1}\leq j_{2}\leq\cdots\leq j_{N})$ are positive integers satisfying $j_{t}<j_{t+1}$ whenever $i_{t}\leq i_{t+1}$. This is a formal power series in $x_{1},x_{2},\ldots$.

For any ${\bf i}={\sf Hecke}(w,N)$, the sequence $(1,2,\ldots,N)$ can be used for ${\bf j}$. Hence,

[TABLE]

S. Fomin-C. Greene [14, Theorem 1.2] states that, up to change of conventions,

[TABLE]

Combining (24) and (25) gives (23).

C. Lenart [27] gave an expression for the symmetric Grothendieck polynomial:

[TABLE]

where $\mu\subseteq\lambda\subseteq\widehat{\mu}$. Here $\widehat{\mu}$ is the unique maximal partition with $t$ rows obtained by adding at most $i-1$ boxes to row $i$ of $\mu$ for $2\leq i\leq t$. In addition, $g_{\mu,\lambda}$ counts the number of Lenart tableaux, i.e., column and row strict tableaux of shape $\mu/\lambda$ with entries in the $i$-th row restricted to $1,2,\ldots,i-1$ for each $i$.

Pick $\mu=M\times M$ and fix $t\geq M^{2}+M$. Therefore $\widehat{\mu}=t\times M$. Hence, by (26),

[TABLE]

Here the sum is over $\mu\subseteq\lambda\subseteq{\widehat{\mu}}$ with $|\lambda|=M^{2}+M$. Now, each such $\lambda$ is of the form $(M\times M,\overline{\lambda})$ where $\overline{\lambda}\in{\sf par}(M)$ and is contained in $M\times M$. Notice that $g_{M\times M,\lambda}\geq f^{\overline{\lambda}}$, since for each such $\lambda$ we can obtain a Lenart-tableau by filling the $\overline{\lambda}$ part with $1,2,\ldots,M$ to obtain a standard tableau, in all possible ways. Hence, using [36, Corollary 7.13.9],

[TABLE]

Finally, let $\pi=M+1,M+2,\ldots,2M,1,2,3\ldots,M\in S_{2M}$. This is a vexillary permutation $\pi$ with $\lambda(\pi)=M\times M$. By, e.g., [24, Lemma 5.4],

[TABLE]

Since the Schur polynomials form a basis of the ring of symmetric polynomials, the righthand sides of (26) and (25) coincide, i.e., $b_{\pi,\lambda}=g_{M\times M,\lambda}$ for every $\lambda$. The result follows. ∎

Example 4.2*.*

Let $w=31524=s_{4}s_{2}s_{3}s_{1}$. Using (23) we obtain

[TABLE]

which the reader may confirm by direct check. ∎

This next generalization of (2) is also manifestly nonnegative. It specializes in the vexillary case in a tantalizing way.

Proposition 4.3**.**

[TABLE]

where $c_{w,\lambda}$ is the number of row and column strict tableaux of shape $\lambda$ whose top to bottom, right to left, column reading word is a Hecke word for $w$. If $w$ is vexillary, then

[TABLE]

Proof.

Work of A. Buch, A. Kresch, M. Shimozono, H. Tamvakis and the third author [10] proves that

[TABLE]

and the latter sum is over all semistandard set-valued tableaux of shape $\lambda$ [8]. Therefore by (24) we have

[TABLE]

proving (28). For the second statement, by [33, Lemma 5.4], when $w$ is vexillary then $G_{w}=G_{\lambda}$, and the above sequence of equalities simplifies, as desired. ∎

Example 4.4*.*

Again let $w=31524$ as in Example 4.2. Now applying (28) gives

[TABLE]

in agreement with Example 4.2. One can check the $f^{\lambda,N}$ computations either directly, or by using

[TABLE]

combined with (26).∎

Proposition 4.3 is our central motivation for Problem 1.4.

Proposition 4.5**.**

Fix $k$. There is an $|\mu|^{O(1)}$ algorithm to compute $f^{\mu,N}$ where $N\leq|\mu|+k$.

Proof.

We use (30) combined with (26) and describe the possible Lenart tableaux. First, we look for $\mu\subseteq\lambda\subseteq\overline{\mu}$ where $|\lambda|=|\mu|+k$. Such $\lambda$ correspond to a choice of $k$ rows $r_{1}\leq r_{2}\leq\ldots\leq r_{k}$ to add a box, among $\ell(\mu)+k-1$ choices (rows $2,3,\ldots,\ell(\mu)+k$). There are ${\ell(\mu)+2k-2\choose k}\in|\mu|^{O(1)}$ many ways to do this. For each such choice, it takes constant time to verify that $\lambda$ is a partition. For those cases, we construct a possible Lenart tableau $T$ by filling row $r_{i}$ in at most $(r_{i}-1)^{k}$ ways. Since $r_{i}\leq\ell(\mu)+k$, there are $|\mu|^{O(1)}$-many possible row strictly increasing tableaux $T$. It remains to determine if $T$ is actually a Lenart tableau, which takes constant time (since $k$ is fixed). Finally, to each tableau, we must compute $f^{\lambda}$ via (3). This takes $|\lambda|^{O(1)}$-time. Now, since $|\lambda|=|\mu|+k$ and $k$ is fixed, it also takes $|\mu|^{O(1)}$-time. Moreover,

[TABLE]

Hence, summing the at most ${\ell(\mu)+2k-2\choose k}$ hook-length calculations, also takes $|\mu|^{O(1)}$-time, as desired. ∎

Example 4.6*.*

We elaborate on the proof of Theorem 4.5 in the case $k=2$. Here we look for $\mu\subseteq\lambda\subseteq\overline{\mu}$ where $|\lambda|=|\mu|+2$. Such $\lambda$ correspond to a choice of two rows $r_{1}\leq r_{2}$ to add a box, among $\ell(\mu)+1$ choices (rows $2,3,\ldots,\ell(\mu)+2$). If $r_{1}=r_{2}$, $g_{\mu,\lambda}={r_{2}-1\choose 2}$. Otherwise if $r_{1}<r_{2}$, there are ${\ell(\mu)+1\choose 2}$ many choices. Assuming $\lambda$ is a partition, there are two cases. If the two boxes are in different columns $g_{\mu,\lambda}=(r_{1}-1)(r_{2}-1)$. Otherwise, if they are in the same column (and hence $r_{2}=r_{1}+1$), then $g_{\mu,\lambda}={r_{2}-1\choose 2}$. Now apply (26).

For $\lambda=\delta_{100}=(100,99,\ldots,3,2,1)$ and $N={100\choose 2}+2$, this procedure exactly computes $f^{\lambda,N}=\#{\sf Hecke}(w_{0},N)=3.75\ldots\times 10^{7981}$. ∎

4.2. Application to Euler characteristics of Brill-Noether varieties (after [2, 11])

Counting standard set-valued tableaux has been given geometric impetus through work of [2, 11] on Brill-Noether varieties. More precisely, following [11, Definition 1.2], let $g,r,d\in{\mathbb{Z}}_{\geq 0}$. Suppose $\alpha=(\alpha_{0}\leq\alpha_{1}\leq\cdots\leq\alpha_{r})$ and $\beta=(\beta_{0}\leq\beta_{1}\leq\cdots\leq\beta_{r})$ be sequences in ${\mathbb{Z}}_{\geq 0}^{r+1}$. Let ${\sf CP}={\sf CP}(g,r,d,\alpha,\beta)$ be the skew Young diagram with boxes

[TABLE]

Example 4.7*.*

If $g=45,d=43,r=3,\alpha=(0,1,1,4),\beta=(0,0,1,3)$ then

[TABLE]

∎

Let $\chi(G^{r,\alpha,\beta}_{d}(X,p,q))$ be the algebraic Euler characteristic of the Brill-Noether variety $G^{r,\alpha,\beta}_{d}(X,p,q)$.

Theorem 4.8** (M. Chan-N. Pfleuger [11]).**

$(-1)^{g-|{\sf CP}|}\chi(G^{r,\alpha,\beta}_{d}(X,p,q))=f^{{\sf CP},g}$.

Given $\lambda/\mu$, construct a ($321$-avoiding) permutation $w_{\lambda/\mu}$ by filling all boxes in the same northwest-southeast diagonal with the same entry, starting with $1$ on the northeastmost diagonal and increasing consecutively as one moves southwest. Call this filling $T_{\lambda/\mu}$. Let $(r_{1},r_{2},\ldots,r_{|\lambda/\mu|})$ be the left-to-right, top-to-bottom, row reading word of $T_{\lambda/\mu}$. Define $w_{\lambda/\mu}=s_{r_{1}}s_{r_{2}}\cdots s_{r_{|\lambda/\mu|}}$.

Example 4.9*.*

If $\lambda/\mu=(12,10,9,9)/(4,3,3,0)$ be ${\sf CP}$ from Example 4.7. Then

[TABLE]

The reading word is

[TABLE]

and $w_{\lambda/\mu}=9,1,2,11,3,12,16,4,5,6,7,8,10,13,14,15\in{S}_{16}$.∎

Earlier, in [2], a cancellative combinatorial formula for $\chi(G^{r,\alpha,\beta}_{d}(X,p,q))$ is given. In [11], another cancellative formula is given, as a signed sum involving counts of (ordinary) skew standard Young tableaux. See [11, Theorem 6.6] (and the discussion of [14] in [11, Section 6]) as well as [2, Theorem A, Theorem C].

Proposition 4.10**.**

Let $w_{\sf CP}$ is a $321$-avoiding permutation defined as above. Then:

[TABLE]

and

[TABLE]

Proof.

By [8, Section 2], $G_{w_{\nu/\mu}}=G_{\nu/\mu}:=\sum_{T}(-1)^{|T|-|\nu/\mu|}{\bf x}^{T}$, where the sum is over semistandard set-valued tableaux of skew shape $\nu/\mu$. Therefore if $f^{\nu/\mu,N}$ is the number of standard set-valued tableaux of this shape with $N$ entries then

[TABLE]

Now combine this with Theorem 4.8, Proposition 4.1 and 4.3. ∎

The second formula expresses $(-1)^{g-|{\sf CP}|}\chi(G^{r,\alpha,\beta}_{d}(X,p,q))$ as a cancellation-free sum of Euler characteristics of other Brill-Noether varieties. Is there a geometric explanation of this?

5. Three importance sampling algorithms

5.1. Estimating $\#{\sf Red}(w)$

Define a random variable ${\sf Y}_{w}$ for $w\in{S}_{n}$, as follows:

if $w$ is vexillary then

${\sf Y}_{w}=f^{\lambda(w)}$

else

C=\{w^{\prime}\text{\ is a child of w$in${\mathcal{T}}(w)}\}

Choose $W^{\prime}\in C$ uniformly at random

${\sf Y}_{w}=\#C\times{\sf Y}_{w^{\prime}}$

Proposition 5.1**.**

Let $w\in{S}_{n}$. Then ${\mathbb{E}}({\sf Y}_{w})=\#{\sf Red}(w)$.

Proof.

We induct on $h=h(w)\geq 0$, the height of ${\mathcal{T}}(w)$, i.e., the maximum length of any path from the root to a leaf. In the base case, $h=0$, $w$ is vexillary and thus, by (5),

[TABLE]

Our induction hypothesis is that ${\mathbb{E}}({\sf Y}_{u})=\#{\sf Red}(u)$ whenever $h(u)<h(w)$. Now

[TABLE]

The last equality is by construction of the transition algorithm and Theorem 2.2. ∎

Example 5.2*.*

Let $w=43817625\in{S}_{8}$. We have the following sequence of transition steps

[TABLE]

The number of children is indicated at each stage. The final permutation is vexillary, and $f^{\lambda(73642158)}=f^{6,4,2,2,1}=243243$. Hence one sample is $3\times 1\times 3\times 2\times 2\times 243243=8756748$. Using sample size $2\times 10^{3}$ gives an estimate of $2.09(\pm 0.04)\times 10^{6}$, versus $\#{\sf Red}(w)=2085655$.222The “$(\pm 0.04)$” refers to the standard error of the mean. All estimates are based on twelve trials of an indicated sample size. Code is available at https://github.com/ICLUE/reduced-word-enumeration∎

Example 5.3* ($w=\sigma^{(n)}=2143\cdots 2n\ 2n-1$).*

When $n=10$ (so $\sigma^{(n)}\in{S}_{20}$), using sample size $10^{5}$ gives an estimate of $3.63(\pm 0.02)\times 10^{6}$, which is close to the exact value $10!=3628800$. When $n=30$ ($\sigma^{(n)}\in{S}_{60}$), using sample size $2\times 10^{6}$ one estimates $2.18(\pm 0.49)\times 10^{32}$ whereas $30!=2.65\ldots\times 10^{32}$. ∎

Example 5.4* (Estimating the number of skew standard Young tableaux).*

We continue Example 4.9. Let $f^{\lambda/\mu}$ be the number of standard Young tableaux of shape $\lambda/\mu$. By a result of S. Billey-W. Jockusch-R. P. Stanley [4, Corollary 2.4], $F_{w_{\lambda/\mu}}=s_{\lambda/\mu}$. Taking the coefficient of $x_{1}x_{2}\cdots x_{|\lambda/\mu|}$ on both sides implies $\#{\sf Red}(w_{\lambda/\mu})=f^{\lambda/\mu}$. One has the textbook determinantal formula

[TABLE]

So $f^{\lambda/\mu}\!=\!73064598262110\!\approx\!7.31\times 10^{13}$. A $10^{4}$ sample size estimate is $7.30(\pm 0.04)\times 10^{13}$. ∎

5.2. Estimating $\#{\sf Hecke}(w,N)$

We propose a different importance sampling algorithm, to compute $\#{\sf Hecke}(w,N)$. For $N<\ell(w)$ the random variable ${\sf Z}_{w,N}$ is equal to [math] and for $N\geq\ell(w)$, it is recursively defined by:

if $w=id$ then

if $N=0$ then  ${\sf Z}_{w,N}=1$  else  ${\sf Z}_{w,N}=0$

else

$D=\{i:w(i)>w(i+1)\}$

Choose $I\in D$ and $\theta\in\{0,1\}$ independently and uniformly at random

if $\theta=0$ then  ${\sf Z}_{w,N}=2\#D\times{\sf Z}_{w,N-1}$  else  ${\sf Z}_{w,N}=2\#D\times{\sf Z}_{ws_{I},N-1}$

Proposition 5.5**.**

Let $w\in{S}_{n}$ and $N\geq\ell(w)$. Then ${\mathbb{E}}({\sf Z}_{w,N})=\#{\sf Hecke}(w,N)$.

Proof.

First we claim

[TABLE]

The unique Hecke word for $w=id$ is the empty word; this explains the first two cases.

Thus assume $w\neq id$ and $N\geq\ell(w)$. Suppose that $(i_{1},i_{2},\ldots,i_{N})\in{\sf Hecke}(w,N)$.

Claim 5.6**.**

$i_{N}$* is the position of a descent of $w$, i.e., $w(i_{N})>w(i_{N}+1)$.*

Proof of Claim 5.6: Consider $w^{\prime}:=s_{i_{1}}\star s_{i_{2}}\star\cdots\star s_{i_{N-1}}$. Either $\ell(w^{\prime})=\ell(w)$ or $\ell(w^{\prime})=\ell(w)-1$. In the former case then if $i_{N}$ is the position of an ascent of $w^{\prime}=w$ then $w=w^{\prime}\star s_{i_{N}}$ would create a descent at that position, a contradiction. In the latter case, $w^{\prime}$ had an ascent at position $i_{N}$ which becomes a descent in $w^{\prime}\star s_{i_{N}}=w^{\prime}s_{i_{N}}$. ∎

Claim 5.6 implies the existence of a bijection

[TABLE]

defined by $(i_{1},i_{2},\ldots,i_{N-1},i_{N})\in{\sf Hecke}(w,N)\mapsto((i_{1},i_{2},\ldots,i_{N-1}),i_{N})$.333 If $N=\ell(w)$, then ${\sf Hecke}(w,N)={\sf Red}(w)$ and ${\sf Hecke}(w,N-1)=\emptyset$. In this case, (34) reduces to the bijection ${\sf Red}(w)\stackrel{{\scriptstyle\sim}}{{\rightarrow}}\bigcup_{i\in D}{\sf Red}(ws_{i})\times\{i\}$. Therefore, by taking cardinalities on both sides of (34) we obtain the third case of (33).

Returning to proposition itself, we induct on $N\geq 0$. The case $N=0$ holds by the first case of (33) and the definition $Z_{w,N}=0$ if $N<\ell(w)$. For $N>0$,

[TABLE]

where we have applied induction (on $N$) and the third case of (33). ∎

Example 5.7*.*

One can explicitly generate all $2030964$ elements of ${\sf Hecke}(351624,13)$. A $2000$ sample size estimate is $2.04(\pm 0.10)\times 10^{6}$. ∎

Example 5.8*.*

By [33, Corollary 1.3],

[TABLE]

For $n=10$, $\#{\sf Hecke}(w_{0},46)=5.65\ldots\times 10^{28}$. Using sample size $10^{8}$, we obtained an estimate of $\approx 4.26(\pm 1.94)\times 10^{28}$. ∎

The ${\sf Z}$-algorithm restricts to an algorithm to compute $\#{\sf Red}(w)$. However, the ${\sf Y}$-algorithm of Subsection 5.1 sometimes has better convergence in this case. This suggests a “hybrid” algorithm. Define ${\sf H}_{w,N}$ to be [math] if $N<\ell(w)$. Otherwise,

if $N=\ell(w)$ then ${\sf H}_{w,N}=Y_{w}$

else if $w=id$ then

if $N=0$ then  ${\sf H}_{w,N}=1$  else ${\sf H}_{w,N}=0$

else

$D=\{i:w(i)>w(i+1)\}$

Choose $I\in D$ and $\theta\in\{0,1\}$ independently and uniformly at random

if $\theta=0$ then  ${\sf H}_{w,N}=2\#D\times{\sf H}_{w,N-1}$  else  ${\sf H}_{w,N}=2\#D\times{\sf H}_{ws_{I},N-1}$

Proposition 5.9**.**

Let $w\in S_{n}$. Then ${\mathbb{E}}[{\sf H}_{w,N}]=\#{\sf Hecke}(w,N)$.

We omit the proof, as it is a straightforward modification of the argument for Proposition 5.5, using Proposition 5.1.

Example 5.10*.*

Let $w=361824795\in S_{9}$; hence $\ell(w)=12$. Using sample size $10^{6}$ with the ${\sf Z}$ algorithm gives $\#{\sf Hecke}(w,25)\approx 5.98(\pm 0.04)\times 10^{16}$. The estimate from the ${\sf H}$ algorithm (with the same sample size) is $\#{\sf Hecke}(w,25)\approx 6.02(\pm 0.08)\times 10^{16}$. For Example 5.8, with $10^{8}$ samples, the H algorithm estimates $\#{\sf Hecke}(w_{0},46)$ as $6.09(\pm 4.69)\times 10^{28}$. ∎

Example 5.11*.*

We use Proposition 4.5 to compute $\#{\sf Hecke}(w_{0},{n\choose 2}+2)$. When $n=7$, $\#{\sf Hecke}(w_{0},23)=2.54\ldots\times 10^{12}$. A $10^{6}$ sample size estimate is $2.60(\pm 0.22)\times 10^{12}$. For $n=10$, $\#{\sf Hecke}(w_{0},47)=6.01\ldots\times 10^{30}$. A $10^{8}$ sample size estimate is $\approx 4.04(\pm 2.17)\times 10^{30}$. ∎

Example 5.12* (Skew set-valued tableaux).*

To estimate $f^{\lambda/\mu,N}$ for $\lambda/\mu=(12,10,9,9)/(4,3,3,0)$ and $N=45$, we use (31) and the ${\sf Z}$-algorithm with sample size $10^{7}$ to predict $f^{\lambda/\mu,45}=\#{\sf Hecke}(w_{\lambda/\mu},45)\approx 1.30(\pm 0.03)\times 10^{33}$. This is backed by the estimate $1.29(\pm 0.06)\times 10^{33}$ using the ${\sf H}$-algorithm with sample size $10^{6}$. We have thus estimated the value of $(-1)^{g-|{\sf CP}|}\chi(G^{r,\alpha,\beta}_{d}(X,p,q))$ for the parameters of Example 4.7. There are a number of ways to theoretically compute this value ([2], [11], Proposition 4.10). What is the exact value? ∎

Acknowledgments

We thank Anshul Adve, David Anderson, Alexander Barvinok, Melody Chan, Yuguo Chen, Anna Chlopecki, Michael Engen, Neil Fan, Sergey Fomin, Sam Hopkins, Allen Knutson, Tejo Nutalapati, Gidon Orelowitz, Colleen Robichaux, Renming Song, John Stembridge, Anna Weigandt and Harshit Yadav for helpful remarks/discussion. We are especially grateful to Brendan Pawlowski for pointing out Theorem 1.1 appears as Theorem 3.2.7 of [31], as well as other remarks. AY was supported by an NSF grant and a Simons Collaboration Grant. This work is part of ICLUE, the Illinois Combinatorics Lab for Undergraduate Experience.