Set-valued Rothe Tableaux and Grothendieck Polynomials

TL;DR

This paper introduces set-valued Rothe tableaux for permutations and demonstrates their use in generating double Grothendieck polynomials for specific permutation classes, extending previous tableau formulas.

Contribution

It defines set-valued Rothe tableaux and shows they can generate double Grothendieck polynomials for 1432-avoiding permutations, generalizing existing formulas.

Findings

Set-valued Rothe tableaux characterize double Grothendieck polynomials for 1432-avoiding permutations.

The formula specializes to known results for 321-avoiding permutations.

Two alternative tableau formulas are derived using tableau complex properties.

Abstract

The notion of set-valued Young tableaux was introduced by Buch in his study of the Littlewood-Richardson rule for stable Grothendieck polynomials. Knutson, Miller and Yong showed that the double Grothendieck polynomials of 2143-avoiding permutations can be generated by set-valued Young tableaux. In this paper, we introduce the structure of set-valued Rothe tableaux of permutations. Given the Rothe diagram $D(w)$ of a permutation $w$, a set-valued Rothe tableau of shape $D(w)$ is a filling of finite nonempty subsets of positive integers into the squares of $D(w)$ such that the rows are weakly decreasing and the columns are strictly increasing. We show that the double Grothendieck polynomials of 1432-avoiding permutations can be generated by set-valued Rothe tableaux. When restricted to 321-avoiding permutations, our formula specializes to the tableau formula for double Grothendieck polynomials due to Matsumura. Employing the properties of tableau complexes given by Knutson, Miller and Yong, we obtain two alternative tableau formulas for the double Grothendieck polynomials of 1432-avoiding permutations.

Equations

$$x\oplus y=x+y-xy.$$

$$m_{ij}(w)=|\{(i,k)\in D(w)|k\le j\}|.$$

$${\mathbf{f}}_0=(1,2,\dots ,n).$$

$${\mathfrak{G}}_w(\mathbf{x},\mathbf{y})=\sum _{T\in \mathrm{SVRT}(w,{\mathbf{f}}_0)}(-1{)}^{|T|-\ell (w)}\prod _{(i,j)\in D(w)}\prod _{t\in T(i,j)}(x_t\oplus y_{m_{ij}(w)+i-t}).$$

$${\mathfrak{G}}_w(\mathbf{x})=\sum _{T\in \mathrm{SVRT}(w,{\mathbf{f}}_0)}(-1{)}^{|T|-\ell (w)}\prod _{(i,j)\in T}\prod _{t\in T(i,j)}{x}_t.$$

$${\mathfrak{S}}_w(\mathbf{x},\mathbf{y})$$

$${\mathfrak{S}}_w(\mathbf{x})$$

$$E_B=\bigcup _{T\in \mathrm{SRT}(w,{\mathbf{f}}_0)}T(i,j).$$

$${\mathfrak{G}}_w(\mathbf{x},\mathbf{y})=\sum _{T\in \mathrm{LSVRT}(w,{\mathbf{f}}_0)}\prod _{B=(i,j)\in D(w)}\prod _{t\in T(i,j)}(x_t\oplus y_{m_{ij}(w)+i-t})$$

$$\cdot \prod _{t\in E_B\setminus T(i,j)}(1-x_t)(1-y_{m_{ij}(w)+i-t}).$$

$${\mathfrak{G}}_w(\mathbf{x},\mathbf{y})=\sum _{T\in \mathrm{SRT}(w,{\mathbf{f}}_0)}\prod _{B=(i,j)\in D(w)}\prod _{t\in T(i,j)}(x_t\oplus y_{m_{ij}(w)+i-t})$$

$$\cdot \prod _{t\in Y_{T,B}}(1-x_t)(1-y_{m_{ij}(w)+i-t}).$$

$$-x\oplus y=-(x\oplus y)=-(x+y-xy).$$

$$G_w(\mathbf{x},\mathbf{y})$$

$$=(-1{)}^{\ell (w)}\sum _{T\in \mathrm{SVRT}(w,{\mathbf{f}}_0)}\prod _{(i,j)\in T}\prod _{t\in T(i,j)}(-x_t\oplus y_{m_{ij}(w)+i-t})$$

$${\lambda }_i=w_{f_k}-k-(f_i-i),{\mu }_i=w_{f_k}-k-(w_{f_i}-i),$$

$${\mathfrak{G}}_w(\mathbf{x},\mathbf{y})=\sum _T(-1{)}^{|T|-\ell (w)}\prod _{\alpha \in \sigma (w)}\prod _{t\in T(\alpha )}(x_t\oplus y_{{\lambda }_{r(\alpha )}+f_{r(\alpha )}-c(\alpha )-t+1}),$$

$${\lambda }_{r(\alpha )}+f_{r(\alpha )}-c(\alpha )+1=m_{ij}(w)+i.$$

$$r(\alpha )$$

$$c(\alpha )$$

$${\lambda }_{r(\alpha )}+f_{r(\alpha )}-c(\alpha )+1$$

$$=j-k+|\{t|w_t>t,w_t>j\}|+i-|\{t|w_t\le t<i\}|$$

$$=j-|\{t|t<w_t\le j\}|-|\{t|w_t\le t<i\}|+i,$$

$$k-|\{t|w_t>t,w_t>j\}|$$

$$=|\{t|t<w_t\le j\}|.$$

$$j-|\{t|t<w_t\le j\}|-|\{t|w_t\le t<i\}|$$

$$=j-(|\{t|t<w_t\le j,t<i\}|+|\{t|w_t\le t<i,w_t\le j\}|)$$

$$=j-|\{t|t<i,w_t\le j\}|=|\{t|t\ge i,w_t\le j\}|$$

$$=|\{(i,k)\in D(w)|k\le j\}|$$

$$=m_{ij}(w).$$

$$\ell (w)=|\{(w_i,w_j)|1\le i<j\le n,w_i>w_j\}|.$$

$${\partial }_{i}f(\mathbf{x})=\frac{f(\mathbf{x})-s_{i}f(\mathbf{x})}{x_i-x_{i+1}},$$

$${\pi }_{i}f(\mathbf{x})={\partial }_i(1-x_{i+1})f(\mathbf{x}).$$

$${\mathfrak{G}}_{w_0}(\mathbf{x},\mathbf{y})=\prod _{i+j\le n}(x_i+y_j-x_i{y}_j).$$

$${\mathfrak{G}}_w(\mathbf{x},\mathbf{y})={\pi }_i{\mathfrak{G}}_{w{s}_i}(\mathbf{x},\mathbf{y}),$$

$${\mathfrak{S}}_{w_0}(\mathbf{x},\mathbf{y})=\prod _{i+j\le n}(x_i-y_j).$$

Full text

1 Introduction

Let $S_{n}$ denote the set of permutations on $\{1,2,\ldots,n\}$. The double Grothendieck polynomials $\mathfrak{G}_{w}(\mathbf{x},\mathbf{y})$ indexed by permutations $w\in S_{n}$ were introduced by Lascoux and Schützenberger [21] as polynomial representatives of the equivariant $K$-theory classes of structure sheaves of Schubert varieties in the flag manifold. These polynomials were originally defined based on the isobaric divided difference operators. Several combinatorial models have been developed to generate $\mathfrak{G}_{w}(\mathbf{x},\mathbf{y})$, see, for example, [8, 12, 16, 15, 22].

On the other hand, tableau formulas for $\mathfrak{G}_{w}(\mathbf{x},\mathbf{y})$ have been found for specific families of permutations. Based on the algebraic geometry of matrix Schubert varieties, Knutson, Miller and Yong [18] showed that for a 2143-avoiding permutation $w$ (also called a vexillary permutation), $\mathfrak{G}_{w}(\mathbf{x},\mathbf{y})$ can be generated by flagged set-valued Young tableaux. A permutation $w=w_{1}w_{2}\cdots w_{n}\in S_{n}$ is 2143-avoiding if there do not exist indices $1\leq i_{1}<i_{2}<i_{3}<i_{4}\leq n$ such that $w_{i_{2}}<w_{i_{1}}<w_{i_{4}}<w_{i_{3}}$. Set-valued Young tableaux were introduced by Buch [7] in his study of the Littlewood-Richardson rule for stable Grothendieck polynomials. Restricting to semistandard Young tableaux (namely, set-valued Young tableaux with each set containing a single integer), the Knutson-Miller-Yong formula specializes to the tableau formula for the Schubert polynomial $\mathfrak{S}_{w}(\mathbf{x})$ of 2143-avoiding permutations due to Wachs [30].

By introducing the structure of tableau complexes and utilizing the tools of commutative algebra, Knutson, Miller and Yong [17] found two other tableau formulas of $\mathfrak{G}_{w}(\mathbf{x},\mathbf{y})$ for 2143-avoiding permutations in terms of semistandard Young tableaux and limit set-valued Young tableaux, respectively. A limit set-valued Young tableau is an assignment of finite nonempty subsets of positive integers to the squares of a Young diagram such that one can pick out an integer from each square to form a semistandard Young tableau.

Recently, Matsumura [24] provided a tableau formula of $\mathfrak{G}_{w}(\mathbf{x},\mathbf{y})$ for 321-avoiding permutations. A permutation $w=w_{1}w_{2}\cdots w_{n}$ is 321-avoiding if there do not exist indices $i_{1}<i_{2}<i_{3}$ such that $w_{i_{1}}>w_{i_{2}}>w_{i_{3}}$. To a 321-avoiding permutation $w$, one can associate a skew Young diagram, denoted $\sigma(w)$. Matsumura [24] showed that for a 321-avoiding permutation $w$, $\mathfrak{G}_{w}(\mathbf{x},\mathbf{y})$ can be generated by flagged set-valued tableaux of shape $\sigma(w)$. This formula generalizes the tableau formula for the single Grothendieck polynomial $\mathfrak{G}_{w}(\mathbf{x})$ of a 321-avoiding permutation given by Anderson, Chen and Tarasca [1]. When restricted to semistandard Young tableaux, it specializes to the formula for the double Schubert polynomial $\mathfrak{S}_{w}(\mathbf{x},\mathbf{y})$ of a 321-avoiding permutation obtained by Chen, Yan and Yang [9].

In this paper, we introduce the structure of set-valued Rothe tableaux. Let $D(w)$ be the Rothe diagram of a permutation $w$. A set-valued Rothe tableau of shape $D(w)$ is a filling of finite nonempty subsets of positive integers into the squares of $D(w)$ such that the sets in each row are weakly decreasing and the sets in each column are strictly increasing. As defined by Buch [7], for two finite nonempty sets $A$ and $B$ of positive integers, $A<B$ if $\max A<\min B$, and $A\leq B$ if $\max A\leq\min B$. It was noticed by Billey, Jockusch and Stanley [6] that when $w$ is a 321-avoiding permutation, $D(w)$ is a skew Young diagram after a reflection about a vertical line. In this case, each row in a set-valued Rothe tableau of shape $D(w)$ is weakly increasing after a reflection about a vertical line, and thus becomes a set-valued Young tableau. Hence set-valued Rothe tableaux can be viewed as a generalization of set-valued Young tableaux from Young diagrams to Rothe diagrams.

Our main objective is to establish set-valued Rothe tableau formulas of $\mathfrak{G}_{w}(\mathbf{x},\mathbf{y})$ for a new family of permutations, namely, 1432-avoiding permutations. A permutation $w=w_{1}w_{2}\cdots w_{n}$ is 1432-avoiding if there do not exist indices $i_{1}<i_{2}<i_{3}<i_{4}$ such that $w_{i_{1}}<w_{i_{4}}<w_{i_{3}}<w_{i_{2}}$. When restricted to 321-avoiding permutations, we show that one of our formulas coincides with the formula of Matsumura [24]. It should be noted that Stankova [28] proved that the number of 1432-avoiding permutations in $S_{n}$ is equal to the number of 2143-avoiding permutations in $S_{n}$.

In order to state our results, we recall some definitions and notation. The Rothe diagram $D(w)$ of a permutation $w\in S_{n}$ can be viewed as a geometric configuration of the inversions of $w$. Consider an $n\times n$ square grid, where we use $(i,j)$ to denote the square in row $i$ and column $j$. Here the rows are numbered from top to bottom and the columns are numbered from left to right. For $1\leq i\leq n$, put a dot in the square $(i,w_{i})$. Then the Rothe diagram $D(w)$ consists of the squares $(i,j)$ such that there is a dot in row $i$ that is to the right of $(i,j)$, and there is a dot in column $j$ that is below $(i,j)$. For example, Figure 1.1(a) is the Rothe diagram of $w=426315$.

As aforementioned, a set-valued Rothe tableau of shape $D(w)$ is a filling of finite nonempty subsets of positive integers into the squares of $D(w)$ such that the rows are weakly decreasing and the columns are strictly increasing. For example, Figure 1.1(b) depicts a set-valued Rothe tableau for $w=426315$. We say that a set-valued Rothe tableau is flagged by a vector $\mathbf{f}=(f_{1},f_{2},\ldots,f_{n})$ of nonnegative integers if every integer in row $i$ does not exceed $f_{i}$. Let ${\rm SVRT}(w,\mathbf{f})$ denote the set of set-valued Rothe tableaux of shape $D(w)$ flagged by $\mathbf{f}$.

For a set-valued Rothe tableau $T$ and a square $B=(i,j)$ of $T$, we use $T(B)$ or $T(i,j)$ to denote the set filled in $B$. Write $|T|=\sum_{B\in D(w)}|T(B)|$. Let $\ell(w)$ denote the length of $w$, or equivalently, $\ell(w)=|D(w)|$. For two variables $x$ and $y$, we adopt the following notation as used by Fomin and Kirillov [12]:

[TABLE]

For a square $(i,j)$ of $D(w)$, define

[TABLE]

Throughout this paper, we use the following specific flag

[TABLE]

Our main result can be stated as follows.

Theorem 1.1

For a permutation $w\in S_{n}$, we have the following equivalent statements.

• (1)

$w$* is a 1432-avoiding permutation.*

• (2)

$\mathfrak{G}_{w}(\mathbf{x},\mathbf{y})$* has the following set-valued Rothe tableau formula:*

[TABLE]

Setting $y_{i}=0$ in (1.2), we obtain a tableau formula for single Grothendieck polynomials of 1432-avoiding permutations.

Corollary 1.2

Let $w\in S_{n}$ be a 1432-avoiding permutation. Then

[TABLE]

The double Schubert polynomial $\mathfrak{S}_{w}(\mathbf{x},\mathbf{y})$ can be obtained from $\mathfrak{G}_{w}(\mathbf{x},\mathbf{y})$ by extracting the monomials of the lowest degree and then replacing $y_{i}$ by $-y_{i}$. Let ${\rm SRT}(w,\mathbf{f})$ be the set of single-valued Rothe tableaux of shape $D(w)$ flagged by $\mathbf{f}$. In other words, ${\rm SRT}(w,\mathbf{f})$ consists of the Rothe tableaux in ${\rm SVRT}(w,\mathbf{f})$ such that the set filled in each square contains exactly one integer. We have the following tableau formulas for double and single Schubert polynomials.

Corollary 1.3

Let $w\in S_{n}$ be a 1432-avoiding permutation. Then

[TABLE]

Furthermore, by introducing the structure of Rothe tableau complexes and employing the properties of tableau complexes given by Knutson, Miller and Yong [17], we also find two alternative tableau formulas of $\mathfrak{G}_{w}(\mathbf{x},\mathbf{y})$ for 1432-avoiding permutations. One is given in terms of single-valued Rothe tableaux, and the other is given in terms of limit set-valued Rothe tableaux. A limit set-valued Rothe tableau is an assignment of finite nonempty subsets of positive integers to the squares of a Rothe diagram such that one can pick out an integer from each square to form a single-valued Rothe tableau. Figure 1.1(c) illustrates a limit set-valued Rothe tableau, where the integers in boldface form a single-valued Rothe tableau.

Let ${\rm LSVRT}(w,\mathbf{f})$ denote the set of limit set-valued Rothe tableaux of shape $D(w)$ flagged by $\mathbf{f}$. Then we have the following two alternative tableau formulas of $\mathfrak{G}_{w}(\mathbf{x},\mathbf{y})$ for 1432-avoiding permutations.

Theorem 1.4

Let $w\in S_{n}$ be a 1432-avoiding permutation.

• (1)

For each square $B=(i,j)\in D(w)$, set

[TABLE]

Then

[TABLE]

• (2)

Given $T\in{\rm SRT}(w,\mathbf{f}_{0})$ and a square $B\in D(w)$, let $Y_{T,B}$ be the set of positive integers $m$ such that $m$ is larger than the (unique) integer in $T(B)$ and replacing the integer in $T(B)$ by $m$ still yields a Rothe tableau in ${\rm SRT}(w,\mathbf{f}_{0})$. Then

[TABLE]

2 Proof of Theorem 1.1

In this section, we aim to prove Theorem 1.1. For simplicity, let

[TABLE]

Denote

[TABLE]

to be the right-hand side of (1.2). We finish the proof of Theorem 1.1 by separately proving the following two theorems.

Theorem 2.1

If $w$ is a 1432-avoiding permutation, then $\mathfrak{G}_{w}(\mathbf{x},\mathbf{y})=G_{w}(\mathbf{x},\mathbf{y})$.

Theorem 2.2

If $w$ contains a 1432 pattern, then $\mathfrak{G}_{w}(\mathbf{x},\mathbf{y})\neq G_{w}(\mathbf{x},\mathbf{y})$.

We use the opportunity here to explain that when $w$ is a 321-avoiding permutation, Theorem 2.1 specializes to the tableau formula for $\mathfrak{G}_{w}(\mathbf{x},\mathbf{y})$ due to Matsumura [24]. To describe the tableau formula in [24], let $f(w)=(f_{1},f_{2},\ldots,f_{k})$ (respectively, $f^{c}(w)=(f_{1}^{c},f_{2}^{c},\ldots,f_{n-k}^{c})$) be the increasing arrangement of the positions $i$ such that $w_{i}>i$ (respectively, $w_{i}\leq i$). Moreover, let $h(w)=(w_{f_{1}},w_{f_{2}},\ldots,w_{f_{k}})$ and $h^{c}(w)=(w_{f_{1}^{c}},w_{f_{2}^{c}},\ldots,w_{f_{n-k}^{c}})$. It can be shown that $w$ is 321-avoiding if and only if the sequences $h(w)$ and $h^{c}(w)$ are both increasing [10]. One may associate a skew shape $\sigma(w)=\lambda/\mu$ to $w$ by letting

[TABLE]

where $1\leq i\leq k$. For a square $\alpha$ of $\sigma(w)$, let $r(\alpha)$ and $c(\alpha)$ denote the row index and the column index of $\alpha$, respectively.

Corollary 2.3** (**Matsumura [24, Theorem 3.1])

Let $w\in S_{n}$ be a 321-avoiding permutation. Then

[TABLE]

where $T$ ranges over set-valued Young tableaux of shape $\sigma(w)$ flagged by $f(w)$.

*Proof.  *We show that for a 321-avoiding permutation $w$, the right-hand side of (2.3) is equal to $G_{w}(\mathbf{x},\mathbf{y})$ as defined in (2). As observed in [6], after deleting the empty rows indexed by $f^{c}(w)$ and the empty columns indexed by $h(w)$ and then reflecting the resulting diagram about a vertical line, $D(w)$ coincides with the above defined skew shape $\sigma(w)$. For example, for $w=312465$, we see that $f(w)=(1,5),f^{c}(w)=(2,3,4,6)$ and $h(w)=(w_{f_{1}},w_{f_{2}})=(3,6)$. So the corresponding shew diagram $\sigma(w)$ is as illustrated in Figure 2.1.

Therefore, each set-valued Rothe tableau $T\in{\rm SVRT}(w,\mathbf{f}_{0})$ can be viewed as a set-valued (skew) Young tableau of shape $\sigma(w)$ flagged by $f(w)$. For a square $(i,j)\in D(w)$, assume that $\alpha$ is the corresponding square of $\sigma(w)$. Then we need to show that

[TABLE]

It is not hard to check that

[TABLE]

Then by (2.2), we have

[TABLE]

where, at the last step, we used the relation

[TABLE]

Since $w$ is 321-avoiding, it is easy to check that if there exists some integer $t$ such that $t<w_{t}\leq j$, then $t<i$. Moreover, if $w_{t}\leq t<i$, then $w_{t}\leq j$. Thus we have

[TABLE]

Combining (2) and (2) yields (2.4). This completes the proof.

2.1 Isobaric divided difference operator

Before proving Theorem 2.1 and Theorem 2.2, we recall some basic definitions. Given a permutation $w=w_{1}w_{2}\cdots w_{n}\in S_{n}$, the length $\ell(w)$ of $w$ is equal to the number of inversions of $w$, namely,

[TABLE]

It is clear that $\ell(w)=|D(w)|$. Let $s_{i}$ denote the simple transposition interchanging $i$ and $i+1$. Notice that $ws_{i}$ is the permutation obtained from $w$ by swapping $w_{i}$ and $w_{i+1}$. The divided difference operator $\partial_{i}$ on the ring $\mathbb{Z}[\mathbf{x}]=\mathbb{Z}[x_{1},x_{2},\ldots,x_{n}]$ of polynomials with integer coefficients is defined by

[TABLE]

where $f(\mathbf{x})\in\mathbb{Z}[\mathbf{x}]$ and $s_{i}f(\mathbf{x})$ is obtained from $f(\mathbf{x})$ by interchanging $x_{i}$ and $x_{i+1}$. One can then define the isobaric divided difference operator $\pi_{i}$ as

[TABLE]

The double Grothendieck polynomial $\mathfrak{G}_{w}(\mathbf{x},\mathbf{y})$ for $w\in S_{n}$ can be defined as follows. For the longest permutation $w_{0}=n\,(n-1)\cdots 1$, set

[TABLE]

For $w\neq w_{0}$, choose a simple transposition $s_{i}$ such that $\ell(ws_{i})=\ell(w)+1$, and let

[TABLE]

where the operator $\pi_{i}$ only acts on the $\mathbf{x}$-variables. Note that (2.7) is independent of the choice of the simple transposition $s_{i}$, since the operator $\pi_{i}$ satisfies the Coxeter relations $\pi_{i}\pi_{i+1}\pi_{i}=\pi_{i+1}\pi_{i}\pi_{i+1}$ and $\pi_{i}\pi_{j}=\pi_{j}\pi_{i}$ for $|i-j|>1$. If we set $y_{i}=0$ for $i\geq 1$, then $\mathfrak{G}_{w}(\mathbf{x},\mathbf{y})$ reduces to the single Grothendieck polynomial $\mathfrak{G}_{w}(\mathbf{x})$.

The double Schubert polynomial $\mathfrak{S}_{w}(\mathbf{x},\mathbf{y})$ can be produced by a similar procedure [20, 23]. Set

[TABLE]

For $w\neq w_{0}$, choose a simple transposition $s_{i}$ such that $\ell(ws_{i})=\ell(w)+1$, and let $\mathfrak{S}_{w}(\mathbf{x},\mathbf{y})=\partial_{i}\mathfrak{S}_{ws_{i}}(\mathbf{x},\mathbf{y})$. By definition, it is easily seen that $\mathfrak{S}_{w}(\mathbf{x},\mathbf{y})$ can be obtained from $\mathfrak{G}_{w}(\mathbf{x},\mathbf{y})$ by taking the lowest degree homogeneous component and then replacing $y_{i}$ by $-y_{i}$ for $i\geq 1$. Analogously, putting $y_{i}=0$, $\mathfrak{S}_{w}(\mathbf{x},\mathbf{y})$ reduces to the single Schubert polynomial $\mathfrak{S}_{w}(\mathbf{x})$. For combinatorial constructions of Schubert polynomials, see for example [2, 3, 4, 5, 6, 11, 13, 14, 19, 31, 32].

2.2 Proof of Theorem 2.1

The key idea is to show that, when $w$ is 1432-avoiding, $G_{w}(\mathbf{x},\mathbf{y})$ is compatible with the isobaric divided difference operator, which allows us to finish the proof by induction. Such an idea was first used by Wachs [30] to establish the tableau formula for the Schubert polynomials of 2143-avoiding permutations. Matsumura [25] and Matsumura and Sugimoto [26] extended this idea to reprove the Knutson-Miller-Yong set-valued tableau formula for the Grothendieck polynomials of 2143-avoiding permutations. Our technique can be viewed as a generalization of that in [25, 26] from Young diagrams to Rothe diagrams.

The longest permutation $w_{0}=n\cdots 21$ is 1432-avoiding. Since $D(w_{0})$ is a staircase Young diagram with $n-i$ squares in row $i$, there is only one tableau $T_{0}$ of shape $D(w_{0})$ flagged by $\mathbf{f}_{0}$, that is, every square in the $i$-th row of $T_{0}$ is filled with $\{i\}$. For each square $(i,j)$ of $D(w_{0})$, one has $m_{ij}(w_{0})=j$. Thus,

[TABLE]

which agrees with $\mathfrak{G}_{w_{0}}(\mathbf{x},\mathbf{y})$.

We now consider a 1432-avoiding permutation $w\neq w_{0}$. Let $r$ be the first ascent of $w$, that is, the smallest position such that $w_{r}<w_{r+1}$. Lemma 2.4 claims that $ws_{r}$ is also 1432-avoiding. Moreover, we will prove that

[TABLE]

see Theorem 2.5. This allows us to give a proof of Theorem 2.1 by induction.

Lemma 2.4

Let $w\neq w_{0}$ be a 1432-avoiding permutation, and $r$ be the first ascent of $w$. Then $ws_{r}$ is a 1432-avoiding permutation.

*Proof.  *Write $w^{\prime}=ws_{r}=w^{\prime}_{1}w^{\prime}_{2}\cdots w^{\prime}_{n}$. Suppose otherwise that $w^{\prime}$ has a subsequence that is order isomorphic to $1432$. Since $w$ is 1432-avoiding and $r$ is the first ascent, any subsequence of $w^{\prime}$ that is order isomorphic to 1432 must be of the form $w^{\prime}_{i}w^{\prime}_{r}w^{\prime}_{r+1}w^{\prime}_{j}$, where $i<r$ and $j>r+1$. Since $w^{\prime}_{i}$ is the smallest element in this subsequence, we have $w^{\prime}_{i}<w^{\prime}_{r+1}$. Noticing that $w^{\prime}_{i}=w_{i}$ and $w^{\prime}_{r+1}=w_{r}$, we see that $w_{i}<w_{r}$. However, since $r$ is the first ascent, we must have $w_{i}>w_{r}$, leading to a contradiction. This completes the proof.

Theorem 2.5

Let $w\neq w_{0}$ be a 1432-avoiding permutation, and $r$ be the first ascent of $w$. Then

[TABLE]

In the rest of this subsection, we present a proof of Theorem 2.5, which can be outlined as follows. We first define an equivalence relation on the two sets ${\rm SVRT}(ws_{r},\mathbf{f}_{0})$ and ${\rm SVRT}(w,\mathbf{f}_{0})$. For an equivalence class $C$ of ${\rm SVRT}(ws_{r},\mathbf{f}_{0})$, let

[TABLE]

denote the polynomial generated by the Rothe tableaux in $C$. In Theorem 2.7, we deduce a formula for $G_{ws_{r}}(C;\mathbf{x},\mathbf{y})$. Similarly, write $G_{w}(C^{\prime};\mathbf{x},\mathbf{y})$ for the polynomial generated by the Rothe tableaux in an equivalence class $C^{\prime}$ of ${\rm SVRT}(w,\mathbf{f}_{0})$. We also obtain an expression for $G_{w}(C^{\prime};\mathbf{x},\mathbf{y})$, see Theorem 2.10. Finally, we establish a bijection $\Phi$ between the set of equivalence classes of ${\rm SVRT}(ws_{r},\mathbf{f}_{0})$ and the set of equivalence classes of ${\rm SVRT}(w,\mathbf{f}_{0})$. The formulas given in Theorems 2.7 and 2.10 allow us to conclude that for any equivalence class $C$ of ${\rm SVRT}(ws_{r},\mathbf{f}_{0})$,

[TABLE]

This leads to a proof of Theorem 2.5.

Unless otherwise stated, we always assume that $w\neq w_{0}$ is a 1432-avoiding permutation, and that $r$ is the first ascent of $w$. For $T\in{\rm SVRT}(ws_{r},\mathbf{f}_{0})$, let

[TABLE]

that is, $E(T)$ is the subset of squares of $T$ containing at least one of $r$ and $r+1$. It should be noted that the definition of $E(T)$ for $T\in{\rm SVRT}(ws_{r},\mathbf{f}_{0})$ depends only on $r$, which has nothing to do with the first ascent of $ws_{r}$.

Definition 2.6

Given two Rothe tableaux $T,T^{\prime}\in{\rm SVRT}(ws_{r},\mathbf{f}_{0})$, we say that $T$ is equivalent to $T^{\prime}$, denoted $T\sim T^{\prime}$, if $E(T)=E(T^{\prime})$ and for every square $B\in D(ws_{r})$,

[TABLE]

The equivalence relation on the set ${\rm SVRT}(w,\mathbf{f}_{0})$ is defined in the same manner. Let ${\rm SVRT}(ws_{r},\mathbf{f}_{0})/\sim$ and ${\rm SVRT}(w,\mathbf{f}_{0})/\sim$ denote the sets of equivalence classes of ${\rm SVRT}(ws_{r},\mathbf{f}_{0})$ and ${\rm SVRT}(w,\mathbf{f}_{0})$, respectively.

Given a Rothe tableau $T$ in ${\rm SVRT}(ws_{r},\mathbf{f}_{0})$ or ${\rm SVRT}(w,\mathbf{f}_{0})$, since the columns of $T$ are strictly increasing, each column of $T$ contains at most two squares in $E(T)$. Let $P(T)$ be the subset of $E(T)$ such that a square $B\in E(T)$ belongs to $P(T)$ if the column of $T$ containing $B$ has only one square (i.e., $B$) in $E(T)$. Let $Q(T)=E(T)\setminus P(T)$, namely, the subset of $E(T)$ such that a square $B\in E(T)$ belongs to $Q(T)$ if the column containing $B$ has exactly two squares in $E(T)$. Evidently, $T\sim T^{\prime}$ if and only if

[TABLE]

Let $C$ be an equivalence class of ${\rm SVRT}(ws_{r},\mathbf{f}_{0})$ or ${\rm SVRT}(w,\mathbf{f}_{0})$, and let $T$ be any given Rothe tableau in $C$. For $i\geq 1$, let $P(T,i)$ be the set of squares of $P(T)$ in row $i$, and let $b_{i}(T)=|P(T,i)|$. Clearly, $P(T,i)$ is empty unless $i\geq r$. Moreover, let

[TABLE]

where $(i,p_{i})$ is the leftmost square in $P(T,i)$. To state the formula for $G_{ws_{r}}(C;\mathbf{x},\mathbf{y})$ or $G_{w}(C;\mathbf{x},\mathbf{y})$, we need to define a polynomial $h(C,i;\mathbf{x},\mathbf{y})$. Set $h(C,i;\mathbf{x},\mathbf{y})=1$ if $b_{i}(T)=0$, and for $b_{i}(T)\geq 1$, let

[TABLE]

Note that $h(C,i;\mathbf{x},\mathbf{y})$ is independent of the choice of the Rothe tableau $T$ in $C$. As will be seen in the proof of Theorem 2.7, $h(C,i;\mathbf{x},\mathbf{y})$ records the contributions of the integers $r$ and $r+1$ in the squares of $P(T,i)$ ($i\geq r+1$) summed over all the Rothe tableaux in $C$.

Theorem 2.7

Let $w\neq w_{0}$ be a 1432-avoiding permutation, and $r$ be the first ascent of $w$. Assume that $C\in{\rm SVRT}(ws_{r},\mathbf{f}_{0})/\sim$ and $T$ is any given Rothe tableau in $C$. Then,

[TABLE]

In the above expression (2.12),

[TABLE]

and

[TABLE]

where $Q^{+}(T)$ denotes the subset of $Q(T)$ consisting of the squares containing $r$.

Although the formula for $G_{ws_{r}}(C,\mathbf{x},\mathbf{y})$ in (2.12) looks a bit complicated, it will be clear from the proof that each factor in (2.12) appears naturally. In fact, for two different Rothe tableaux $T,T^{\prime}\in C$, $T$ and $T^{\prime}$ can be possibly different only in the squares of $P(T)$. Thus the first factor is the contribution of the integers other than $r$ and $r+1$. We shall show that the second factor is the contribution of $r$ in $P(T,r)$, $J_{C}(\mathbf{x},\mathbf{y})$ is the contribution of $r$ and $r+1$ in $Q(T)$, and $H_{C}(\mathbf{x},\mathbf{y})$ is the contribution of $r$ and $r+1$ in $P(T,i)$ with $i\geq r+1$ summed over all the Rothe tableaux in $C$.

To prove Theorem 2.7, we need two lemmas concerning the configuration of the squares in the sets $P(T)$ and $Q(T)$.

Lemma 2.8

Let $w\neq w_{0}$ be a 1432-avoiding permutation. Assume that $T$ is a Rothe tableau in ${\rm SVRT}(w,\mathbf{f}_{0})$ or ${\rm SVRT}(ws_{r},\mathbf{f}_{0})$, and $(i,j)\in P(T)$. Then there do not exist two squares $(i,k),(h,k)\in Q(T)$ such that $k>j$ and $h<i$.

*Proof.  *We only give a proof for the case when $T\in{\rm SVRT}(w,\mathbf{f}_{0})$. The same argument applies to the case when $T\in{\rm SVRT}(ws_{r},\mathbf{f}_{0})$. Suppose to the contrary that there exist two squares $(i,k),(h,k)\in Q(T)$ such that $k>j$ and $h<i$, see Figure 2.2 for an illustration.

Since $(i,j),(h,k)\in D(w)$, it follows that $w_{h}>j$ and $j$ appears after $w_{h}$ in $w$. This implies that $(h,j)\in D(w)$. Keep in mind that each of the sets $T(i,j)$, $T(i,k)$ and $T(h,k)$ contains at least one of the integers $r$ and $r+1$. Since the rows of $T$ are weakly decreasing and the columns of $T$ are strictly increasing, we see that $r\in T(h,k)$, $r+1\in T(i,j)$. This forces that $T(h,j)=\{r\}$, and hence $(i,j)\in Q(T)$, which contradicts the assumption that $(i,j)\in P(T)$.

It should be noted that Lemma 2.8 is valid for any permutation since the pattern avoidance condition is not required in the proof.

Lemma 2.9

Let $w\neq w_{0}$ be a 1432-avoiding permutation. Assume that $T$ is a Rothe tableau in ${\rm SVRT}(w,\mathbf{f}_{0})$ or ${\rm SVRT}(ws_{r},\mathbf{f}_{0})$, and $(i,j)\in P(T)$. If $i>r$, then there do not exist two squares $(i,k),(h,k)\in Q(T)$ such that $h>i$ and $k<j$.

*Proof.  *We only give a proof for $T\in{\rm SVRT}(w,\mathbf{f}_{0})$, and the arguments for $T\in{\rm SVRT}(ws_{r},\mathbf{f}_{0})$ can be carried out in the same manner. Suppose otherwise that there exist two squares $(i,k)$ and $(h,k)$ in $Q(T)$ where $i<h$ and $j>k$, as illustrated in Figure 2.3.

Notice that both the sets $T(i,j)$ and $T(i,k)$ contain $r$, while the set $T(h,k)$ contains $r+1$. We have the following two claims.

Claim 1: $w_{s}<k$ for any $i<s<h$. Suppose otherwise that there exists some $i<s<h$ such that $w_{s}>k$. Then the square $(s,k)$ belongs to $D(w)$. Since $r\in T(i,k)$ and $r+1\in T(h,k)$, it follows that $r<\min T(s,k)<r+1$, leading to a contradiction.

Claim 2: $k<w_{h}\leq j$. Since $(h,k)$ is a square in $D(w)$, it is clear that $k<w_{h}$. Suppose otherwise that $w_{h}>j$. It follows from Claim 1 that $j$ must appear in $w$ after the position $h$. This implies that $(h,j)\in D(w)$. Since $r\in T(i,j)$ and $r+1\in T(h,k)$, we must have $T(h,j)=\{r+1\}$. This implies that $(i,j)\in Q(T)$, contradicting the assumption that $(i,j)\in P(T)$.

By Claim 2 and the fact that $w_{i}>j$, we see that $w_{i}w_{h}k$ forms a decreasing subsequence of $w$. Since $w$ is 1432-avoiding, we have $w_{t}>k$ for any $1\leq t<i$. Thus, for any $1\leq t<i$, the square $(t,k)$ belongs to $D(w)$. Keep in mind that each integer in row $i$ of $T$ cannot exceed $i$ and the columns of $T$ are strictly increasing. So we have $T(t,k)=\{t\}$ for $1\leq t\leq i$. In particular, we have $T(i,k)=\{i\}$. Since $r\in T(i,k)$, we must have $i=r$, contradicting the assumption that $i>r$. This completes the proof.

Based on Lemmas 2.8 and 2.9, we can now give a proof of Theorem 2.7.

Proof of Theorem 2.7. Assume that $T^{\prime}\in{\rm SVRT}(ws_{r},\mathbf{f}_{0})$ is a Rothe tableau in the equivalence class $C$. Then $T^{\prime}$ differs from $T$ only possibly in the squares of $P(T)$. Note that if $P(T,i)$ is nonempty, then we must have $i\geq r$. Moreover, since the integers appearing in $r$-th row of $T^{\prime}$ cannot exceed $r$, it follows that for any square $B\in P(T,r)$, $T^{\prime}(B)$ does not contain $r+1$. Thus, for $B\in P(T,r)$, $r\in T(B)=T^{\prime}(B)$ and $r+1\notin T(B)=T^{\prime}(B)$.

Before we proceed, we give an illustration of the configuration of the squares in the first $r+1$ rows of $D(w)$ and $D(ws_{r})$, which will be helpful to analyze the contributions of the integer $r$ in the squares of $P(T,r)$. Notice that $D(w)$ is obtained from $D(ws_{r})$ by deleting the square $(r,w_{r})$ and then moving each square in row $r$, that lies to the right of $(r,w_{r})$, down to row $r+1$. Since $r$ is the first ascent of $w$, the first $r+1$ rows of $D(w)$ and $D(ws_{r})$ are as depicted in Figure 2.4, where the square $(r,w_{r})$ of $D(ws_{r})$ is signified by a symbol $\heartsuit$.

Obviously, the first $w_{r}-1$ squares in the $r$-th row (respectively, ($r+1$)-th row) of $T$ are filled with the set $\{r\}$ (respectively, $\{r+1\}$). This implies that each square in the ($r+1$)-th row of $D(ws_{r})$ belongs to $Q(T)$ and the set $P(T,r+1)$ is empty. Therefore, the contribution of the $r$’s in squares of $P(T,r)$ to $G_{ws_{r}}(C;\mathbf{x},\mathbf{y})$ is

[TABLE]

On the other hand, the contribution of the $r$’s and $(r+1)$’s in squares of $P(T,i)$ for $i>r+1$ to $G_{ws_{r}}(C;\mathbf{x},\mathbf{y})$ is

[TABLE]

where $P^{+}(T^{\prime},i)$ (respectively, $P^{-}(T^{\prime},i)$) denotes the subset of $P(T^{\prime},i)$ consisting of squares containing $r$ (respectively, $r+1$). Moreover, the contribution of the $r$’s and $(r+1)$’s in squares of $Q(T)$ to $G_{ws_{r}}(C;\mathbf{x},\mathbf{y})$ is

[TABLE]

where $Q^{+}(T)$ (respectively, $Q^{-}(T)$) denotes the subset of $Q(T)$ consisting of the squares containing $r$ (respectively, $r+1$). Consequently, we obtain that

[TABLE]

Comparing (2.16) with (2.12), in order to complete the proof, we need to show that

[TABLE]

Let us first prove (2.17). To this end, we show that if there are two squares $(r,j_{1})$ and $(r,j_{2})$ in $P(T,r)$ with $j_{1}<j_{2}$ and there exists a square $(r,j)\in D(ws_{r})$ for some $j_{1}<j<j_{2}$, then $(r,j)\in P(T,r)$. It suffices to prove the following claim.

Claim. For $(r,j)\in P(T,r)$, there do not exist squares $(r,k),(h,k)\in Q(T)$ such that $h>r+1$ and $k<j$.

To verify this claim, we construct a Rothe tableau $\overline{T}$ from $T$ such that $\overline{T}\in{\rm SVRT}(w,\mathbf{f}_{0})$. Let $R$ be the set of squares of $D(ws_{r})$ in row $r$ that are strictly to the right of $(r,w_{r})$. Define $\overline{T}$ to be the tableau obtained from $T$ by deleting the square $(r,w_{r})$ together with $T(r,w_{r})$, and then moving each square $B$ in $R$, together with $T(B)$, down to row $r+1$. By construction, it is easy to check that $\overline{T}\in{\rm SVRT}(w,\mathbf{f}_{0})$. Note that $(r,j)\in P(T)$ if and only if $(r+1,j)\in P(\overline{T})$. Applying Lemma 2.9 to $\overline{T}$, we see that if $(r+1,j)\in P(\overline{T})$, then there do not exist squares $(r,k),(h,k)\in Q(\overline{T})$ with $h>r+1$ and $k<j$. Since $Q(T)=Q(\overline{T})$, we conclude the claim.

By the above claim, the configuration of the squares of $P(T)$ and $Q(T)$ in the $r$-th row of $D(ws_{r})$ is as illustrated in Figure 2.5, where the squares in $Q^{+}(T)$ (respectively, $Q^{-}(T)$) are marked with a $\ast$ (respectively, $\star$). In view of the definition $m_{i,j}(ws_{r})$ in (1.1) as well as the definition $\ell_{r}(T)$ in (2.10), we see that (2.17) holds.

We next prove (2.18). For $i>r$, by Lemma 2.8 and Lemma 2.9, the configuration of the squares of $P(T)$ and $Q(T)$ must be as illustrated as in Figure 2.6. In particular, every square in row $i$ of $D(ws_{r})$ that lies between the leftmost square and the rightmost square of $P(T,i)$ must belong to $P(T,i)$.

Assume that $T^{\prime}$ is a Rothe tableau in $C$. Keep in mind that $P(T^{\prime},i)=P(T,i)$. Then we have the following two cases.

Case 1: In $T^{\prime}$, the first $k$ $(0\leq k\leq b_{i}(T))$ squares in $P(T,i)$ contain $r+1$, and the remaining $b_{i}(T)-k$ squares in $P(T,i)$ contain $r$. In this case, running over the Rothe tableaux $T^{\prime}$ in $C$, the integers $r$ and $r+1$ in $P(T,i)$ contribute

[TABLE]

Case 2: In $T^{\prime}$, the first $k-1$ $(1\leq k\leq b_{i}(T))$ squares in $P(T,i)$ contain $r+1$, the $k$-th square contains both $r$ and $r+1$, and the remaining $b_{i}(T)-k$ squares in $P(T,i)$ contain $r$. In this case, running over the Rothe tableaux $T^{\prime}$ in $C$, the integers $r$ and $r+1$ in $P(T,i)$ contribute

[TABLE]

Combining (2.20) and (2.21), we see that Case 1 and Case 2 together contribute the factor $h(C,i;\mathbf{x},\mathbf{y})$ as defined in (2.11) to the summation $F_{C}(\mathbf{x},\mathbf{y})$ in (2.14). Running over the row indices $i$ with $i>r+1$ yields (2.18).

Finally, we verify (2.19). For each $(i,j)\in Q^{+}(T)$, we use $(i^{\prime},j)$ to denote the square in $Q^{-}(T)$ that lie in the same column as $(i,j)$. Then we have

[TABLE]

Write $w^{\prime}=ws_{r}$. We assert that $w^{\prime}_{t}<j$ for $i<t<i^{\prime}$. Suppose otherwise that $w^{\prime}_{t}>j$. Since the square $(i^{\prime},j)\in D(ws_{r})$, we see that $(t,j)\in D(ws_{r})$. Thus we have $r<\min T(t,j)\leq\max T(t,j)<r+1$, leading to a contradiction. This verifies the assertion. By the definition of $m_{ij}(w)$ in (1.1), it is easy to see that

[TABLE]

Therefore, by the above assertion, we obtain

[TABLE]

and so that

[TABLE]

Putting (2.23) into (2.22), we arrive at the equality in (2.19). This completes the proof.

The following theorem provides a formula for the polynomial $G_{w}(C^{\prime};\mathbf{x},\mathbf{y})$ for an equivalence class $C^{\prime}\in{\rm SVRT}(w,\mathbf{f}_{0})/\sim$.

Theorem 2.10

Let $w\neq w_{0}$ be a 1432-avoiding permutation, and $r$ be the first ascent of $w$. Assume that $C^{\prime}\in{\rm SVRT}(w,\mathbf{f}_{0})/\sim$ and $T^{\prime}$ is any given Rothe tableau in $C^{\prime}$. Then,

[TABLE]

where

[TABLE]

and

[TABLE]

Sketch of the proof. The proof is nearly the same as the arguments for Theorem 2.7. The only difference is to notice that $P(T^{\prime},r)$ is empty and that the squares in $P(T^{\prime},r+1)$ contributes the factor $h(C^{\prime},r+1;\mathbf{x},\mathbf{y})$.

To finish the proof of Theorem 2.5, we need a one-to-one correspondence $\Phi$ between the two sets of equivalence classes:

[TABLE]

Construction of the bijection $\Phi$: Assume that $C\in{\rm SVRT}(ws_{r},\mathbf{f}_{0})/\sim$ and $T\in C$. Let $T^{\prime}=\overline{T}\in{\rm SVRT}(w,\mathbf{f}_{0})$ be the Rothe tableau as constructed in the proof of (2.17). That is, $T^{\prime}$ is the Rothe tableau obtained from $T$ by deleting the square $(r,w_{r})$ together with $T(r,w_{r})$, and then moving each square $B$ in $R$, together with $T(B)$, down to row $r+1$, where $R$ is the set of squares of $D(ws_{r})$ in row $r$ that are strictly to the right of $(r,w_{r})$. Let $C^{\prime}\in{\rm SVRT}(w,\mathbf{f}_{0})/\sim$ be the equivalence class containing $T^{\prime}$. It is clear that $C^{\prime}$ is independent of the choice of $T$. Set $\Phi(C)=C^{\prime}$.

The inverse of $\Phi$ can be described as follows. Let $C^{\prime}\in{\rm SVRT}(w,\mathbf{f}_{0})/\sim$ and $T^{\prime}\in C^{\prime}$. Let $T^{\prime\prime}$ be the Rothe tableau defined by setting $T^{\prime\prime}(B)=T^{\prime}(B)$ if $B\in D(w)\setminus P(T^{\prime},r+1)$, and setting

[TABLE]

Notice that $T^{\prime\prime}\in{\rm SVRT}(w,\mathbf{f}_{0})$. We define $T$ as the Rothe tableau obtained from $T^{\prime\prime}$ by adding the square $(r,w_{r})$ filled with the set $\{r\}$, and then moving each square $B$ of $T^{\prime\prime}$ (together with the set $T^{\prime\prime}(B)$), which is to the right of the square $(r+1,w_{r})$, up to row $r$. By construction, it is easily checked that $T\in{\rm SVRT}(ws_{r},\mathbf{f}_{0})$. Let $C$ be the equivalence class in ${\rm SVRT}(ws_{r},\mathbf{f}_{0})/\sim$ containing $T$. Set $\Phi^{-1}(C^{\prime})=C$.

Based on Theorem 2.7 and Theorem 2.10, we can establish the following relation.

Theorem 2.11

Let $w\neq w_{0}$ be a 1432-avoiding permutation, and $r$ be the first ascent of $w$. For each equivalence class $C\in{\rm SVRT}(ws_{r},\mathbf{f}_{0})/\sim$, we have

[TABLE]

The proof of Theorem 2.11 requires the following property concerning the operator $\pi_{r}$ due to Matsumura [24].

Lemma 2.12** (**Matsumura [24, Lemma 4.1])

For an arbitrary sequence $(a_{1},a_{2},\ldots,a_{m})$ of positive integers,

[TABLE]

Furthermore, the expression in (2.26) is symmetric in $x_{r}$ and $x_{r+1}$.

Proof of Theorem 2.11. Assume that $T$ is any given Rothe tableau in $C$. The polynomial $h(C,i;\mathbf{x},\mathbf{y})$ defined in (2.11) has the following reformulation:

[TABLE]

Hence $(-1)^{b_{i}(T)}h(C,i;\mathbf{x},\mathbf{y})$ coincides with the right-hand side of (2.26) by setting $m=b_{i}(T)+1$ and setting $a_{j}=\ell_{i}(T)+j-1$ for $1\leq j\leq m$, and then exchanging the variables $x_{r}$ and $x_{r+1}$. It follows from Lemma 2.12 that

[TABLE]

which is a symmetric polynomial in $x_{r}$ and $x_{r+1}$.

On the other hand, if a polynomial $f(\mathbf{x})$ is symmetric in $x_{r}$ and $x_{r+1}$, then for any polynomial $g(\mathbf{x})$, it is easily checked that

[TABLE]

Therefore, applying $\pi_{r}$ to the formula of $G_{ws_{r}}(C;\mathbf{x},\mathbf{y})$ in Theorem 2.7, we obtain that

[TABLE]

Let $T^{\prime}\in\Phi(C)$ be any given Rothe tableau in the equivalent class of $\Phi(C)$. By the construction of $\Phi$, it is easy to see that

[TABLE]

Again, by the construction of $\Phi$, it is also clear that for $i>r+1$,

[TABLE]

which imply that

[TABLE]

Moreover, since $Q(T)=Q(T^{\prime})$ and $m_{ij}(ws_{r})=m_{ij}(w)$ for any $(i,j)\in Q^{+}(T)$, one has

[TABLE]

Still, by the construction of $\Phi$, we see that

[TABLE]

So, by (2.2), we have

[TABLE]

Substituting (2.29)–(2.2) into (2.2), we see that $\pi_{r}G_{ws_{r}}(C;\mathbf{x},\mathbf{y})=G_{w}(\Phi(C);\mathbf{x},\mathbf{y})$. This completes the proof.

By Theorem 2.11 and the bijection $\Phi$, we arrive at a proof of Theorem 2.5. Using induction on the length of $w$, we reach a proof of Theorem 2.1.

2.3 Proof of Theorem 2.2

In this subsection, we confirm Theorem 2.2 by proving the following statement.

Theorem 2.13

If $w$ contains a 1432 pattern, then

[TABLE]

By Corollary 1.3, if $w$ is a 1432-avoiding permutation, then $\mathfrak{S}_{w}(\mathbf{x})$ must equal the right-hand side of (2.33). Hence Theorem 2.13 implies Theorem 2.2.

To finish the proof of Theorem 2.13, we recall the balanced labeling model of Schubert polynomials given by Fomin, Greene, Reiner and Shimozono [11]. To a square $(i,j)$ in the Rothe diagram $D(w)$, the associated hook $H_{i,j}(w)$ is the collection of squares $(i^{\prime},j^{\prime})$ of $D(w)$ such that either $i^{\prime}=i$ and $j^{\prime}\geq j$, or $i^{\prime}\geq i$ and $j^{\prime}=j$.

A labeling $L$ of $D(w)$ is an assignment of positive integers into the squares of $D(w)$ such that each square receives exactly one integer. We use $L(i,j)$ to denote the label in the square $(i,j)\in D(w)$. A labeling $L$ is called balanced if for every square $(i,j)\in D(w)$, the label $L(i,j)$ remains unchanged after rearranging the labels in the hook $H_{i,j}(w)$ so that they are weakly increasing from right to left and from top to bottom. Figure 2.7 illustrates two balanced labelings for the permutation $w=25143$.

A balanced labeling of $D(w)$ is said to be column strict if no column contains two equal labels. Let ${\rm CSBL}(w,\mathbf{f}_{0})$ denote the set of column strict balanced labelings of $D(w)$ such that $L(i,j)\leq i$ for each square $(i,j)\in D(w)$. Fomin, Greene, Reiner and Shimozono [11] showed that

[TABLE]

We are now in a position to give a proof of Theorem 2.13.

Proof of Theorem 2.13. Assume that $w$ is a permutation that contains a pattern 1432. Recall that ${\rm SRT}(w,\mathbf{f}_{0})$ is the set of single-valued Rothe tableaux of shape $D(w)$ flagged by $\mathbf{f}_{0}$. By definition, it is clear that a Rothe tableau in ${\rm SRT}(w,\mathbf{f}_{0})$ is a column strict balanced labeling, and hence belongs to ${\rm CSBL}(w,\mathbf{f}_{0})$. In view of (2.34), to prove (2.33), it suffices to show that there exits a balanced labeling in ${\rm CSBL}(w,\mathbf{f}_{0})$ that does not belong to ${\rm SRT}(w,\mathbf{f}_{0})$. We next construct such a balanced labeling $L$ in ${\rm CSBL}(w,\mathbf{f}_{0})$.

Suppose that the subsequence $w_{i_{1}}w_{i_{2}}w_{i_{3}}w_{i_{4}}$ of $w$ has the same relative order as the pattern 1432, that is, $w_{i_{1}}<w_{i_{4}}<w_{i_{3}}<w_{i_{2}}$. Since $w_{i_{3}}>w_{i_{4}}$, there exists at least one square in the $i_{3}$-th row of $D(w)$. Let $(i_{3},j)$ be the rightmost square in this row. Let

[TABLE]

be the subset of $D(w)$ consisting of the squares in column $j$ lying between row $i_{1}$ and row $i_{3}$. We classify $S$ into two subsets according to whether a square $(i,j)\in S$ is the rightmost square in the row or not. Let $S_{1}\subseteq S$ consists of square $(i,j)\in S$ such that $(i,j)$ is the rightmost square in row $i$. Clearly, $S_{1}$ is nonempty since it contains the square $(i_{3},j)$. Let $S_{2}=S\backslash S_{1}$ be the complement. Since $w_{i_{2}}>w_{i_{3}}$, we see that the two squares $(i_{2},j),(i_{2},w_{i_{3}})$ belong to $D(w)$. Hence $(i_{2},j)\in S_{2}$, and so $S_{2}$ is also nonempty.

Let us use an example in Figure 2.8 to illustrate the sets $S_{1}$ and $S_{2}$. In this example, $w=1\,4\,5\,9\,6\,10\,7\,8\,2\,3$ and the subsequence $w_{1}w_{6}w_{7}w_{9}$ forms a 1432-pattern. The rightmost square of $D(w)$ in the $i_{3}$-th row is the square $(7,3)$, and so we have

[TABLE]

Moreover, the squares belonging to $S_{1}$ and $S_{2}$ are signified with $\spadesuit$ and $\clubsuit$ in Figure 2.8(a), respectively.

Let $i_{0}$ be the smallest row index such that: (1) the square $(i_{0},j)\in S_{1}$; (2) there exists a square in $S_{2}$ lying above $(i_{0},j)$. Such an row index exists since the $i_{3}$-th row satisfies the above conditions. Let $S^{\prime}=\{(i,j)\in S_{1}\,|\,i\leq i_{0}\}$ be the subset of $S_{1}$ including the squares above $(i_{0},j)$. In the example in Figure 2.8, we see that $i_{0}=5$ and the squares of $S^{\prime}$ are signified with the symbol $\diamondsuit$.

Assume that $|S^{\prime}|=k$ and $(r_{1},j),\ldots,(r_{k},j)$ are the squares of $S^{\prime}$, where $r_{1}<\cdots<r_{k}=i_{0}$. Note that $r_{1}>i_{1}$. This is because $w_{i_{1}}$ is the smallest element of $\{w_{i_{1}},w_{i_{2}},w_{i_{3}},w_{i_{4}}\}$ and thus the square $(i_{1},j)\notin D(w)$.

We now construct a balanced labeling $L$ of $D(w)$ as follows. If a square $(s,t)$ of $D(w)$ is not contained in $S^{\prime}$, then we set $L(s,t)=s$. For the squares $(r_{1},j),\ldots,(r_{k},j)$ of $S^{\prime}$, we set $L(r_{1},j)=i_{1}$ and $L(r_{p},j)=r_{p-1}$ for $p=2,\ldots,k$. For the permutation in Figure 2.8, the labeling $L$ is given in Figure 2.9, where the integers in $S^{\prime}$ are written in boldface.

By the construction of $L$, it is not hard to check that $L$ is a column strict balanced labeling in ${\rm CSBL}(w,\mathbf{f}_{0})$. Moreover, the entries in the $j$-th column of $L$ are not increasing. So $L$ does not belong to ${\rm SRT}(w,\mathbf{f}_{0})$. This completes the proof.

3 Rothe tableau complexes

In this section, we prove the tableau formulas in Theorem 1.4. To do this, we investigate the structure of Rothe tableau complexes, which is a specific family of the tableau complexes as introduced by Knutson, Miller and Yong [17]. Using Theorem 1.1 and the properties of tableau complexes established in [17], we obtain two alternative tableau formulas for the Grothendieck polynomials of 1432-avoiding permutations, as given in Theorem 1.4.

Let us proceed with a brief review of the Hilbert series of the Stanley-Reisner ring of a simplicial complex, see [27, 29] for more detailed information. An (abstract) simplicial complex $\Delta$ on a finite vertex set $V$ is a collection of subsets of $V$ such that if $\sigma\in\Delta$ and $\tau\subseteq\sigma$, then $\tau\in\Delta$. Each subset $\sigma\in\Delta$ is called a face of $\Delta$. A face $\sigma$ is called a facet of $\Delta$ if $\sigma$ is not a subset of any other faces. Clearly, $\Delta$ is determined by its facets.

Let $\mathds{k}[\mathbf{t}]$ be the ring of polynomials over a field $\mathds{k}$ in the variables $t_{v}$ where $v\in V$. The Stanley-Reisner ideal $I_{\Delta}$ is the ideal generated by the monomials corresponding to the subsets of $V$ that are not faces of $\Delta$, namely,

[TABLE]

The Stanley-Reisner ring of $\Delta$, denoted $\mathds{k}[\Delta]$, is the quotient ring $\mathds{k}[\mathbf{t}]/I_{\Delta}$. The Hilbert series $H(\mathds{k}[\Delta];\mathbf{t})$ of $\mathds{k}[\mathbf{t}]/I_{\Delta}$ is equal to the sum of monomials not belonging to $I_{\Delta}$. It is well known [27, 29] that $H(\mathds{k}[\Delta];\mathbf{t})$ has the following formula:

[TABLE]

where

[TABLE]

The numerator $K(\mathds{k}[\Delta];\mathbf{t})$ is called the $K$-polynomial of $\mathds{k}[\Delta]$.

We now restrict attention to the $K$-polynomials of tableau complexes introduced in [17]. Let $X$ and $Y$ be two finite sets. A map $f$ from $X$ to $Y$ is called a tableau, which can be viewed as an assignment of elements of $Y$ to elements of $X$ such that each $x\in X$ receives exactly one element of $Y$. A tableau $f$ can also be identified with the following set

[TABLE]

of ordered pairs. Let $U$ be a subset of tableaux from $X$ to $Y$, and let $E\subseteq X\times Y$ be a set of ordered pairs such that $f\subseteq E$ for each $f\in U$. The tableau complex corresponding to $U$ and $E$, denoted $\Delta_{E}(X\xrightarrow{U}Y)$, can be defined as follows. Let us first define a simplex $\Delta_{E}$. For each pair $(x\mapsto a)\in E$, write $(x\arrownot\mapsto y)=E\setminus\{(x\mapsto y)\}$ for the complement of $\{(x\mapsto y)\}$, and let

[TABLE]

Denote by $\Delta_{E}$ the simplex with vertex set $V$, that is, $\Delta_{E}$ is the collection of all of the subsets of $V$.

Let $F\subseteq V$ be a face of $\Delta_{E}$. Assume that $F$ has $k$ vertices $(x_{1}\arrownot\mapsto y_{1}),\ldots,(x_{k}\arrownot\mapsto y_{k})$. Then $F$ can be identified with the following subset of $E$:

[TABLE]

On the other hand, each subset of $E$ can be viewed as a set-valued tableau from $X$ to $Y$, that is, a map that assigns each element of $X$ with a subset of $Y$. To be more specific, for a subset $A$ of $E$, the corresponding set-valued tableau is defined by assigning $x\in X$ with the subset $\{y\in Y\,|\,(x\mapsto y)\in A\}$. So the face $F$ of $\Delta_{E}$ can also be identified with a set-valued tableau such that for $x\in X$,

[TABLE]

From now on, a face $F$ of $\Delta_{E}$ can be identified either with a subset of $E$ or with a set-valued tableau from $X$ to $Y$, which will not cause confusion from the context. By the definition of $\Delta_{E}$, a vertex $(x\arrownot\mapsto y)\in V$ belongs to $F$ if and only if the pair $(x\mapsto y)$ does not belong to $F$.

Recall that $U$ is a set of tableaux from $X$ to $Y$ such that $f\subseteq E$ for each $f\in U$. So each tableau $f$ in $U$ is a face of $\Delta_{E}$. The tableau complex $\Delta_{E}(X\xrightarrow{U}Y)$ is defined as the subcomplex of $\Delta_{E}$ such that the facets of $\Delta_{E}(X\xrightarrow{U}Y)$ are the tableaux in $U$. This means that a set-valued tableau $F\subseteq E$ is a face of $\Delta_{E}(X\xrightarrow{U}Y)$ if and only if $F$ contains some tableau $f\in U$.

When $X$ and $Y$ are further endowed with partially ordered structures, Knutson, Miller and Yong [17] found three different expressions for the $K$-polynomial of a tableau complex.

Theorem 3.1** (**Knutson-Miller-Yong [17])

Let $X$ and $Y$ be two finite posets. For each $x\in X$, let $Y_{x}$ be a totally ordered subset of $Y$. Let $\Psi$ be a set of pairs $(x,x^{\prime})$ in $X$ with $x<x^{\prime}$. Let $U$ be the set of tableaux $f\colon X\rightarrow Y$ such that

• (a)

$f(x)\in Y_{x}$;

• (b)

$f$* is weakly order preserving, that is, if $x\leq x^{\prime}$, then $f(x)\leq f(x^{\prime})$;*

• (c)

if $(x,x^{\prime})\in\Psi$, then $f(x)<f(x^{\prime})$.

Set $E=\bigcup_{f\in U}f$. Let $\mathbf{t}=\{t_{(x\arrownot\mapsto a)}\,|\,(x\arrownot\mapsto a)\in V\}$ be the set of variables corresponding to the vertices of the tableau complex $\Delta=\Delta_{E}(X\xrightarrow{U}Y)$. Then, $\Delta$ is homeomorphic to a ball or a sphere. Moreover, the corresponding $K$-polynomial has the following expressions.

Let $U_{1}$ be the set of set-valued tableaux $F\subseteq E$ such that every tableau $f\subseteq F$ lies in $U$. Then,

[TABLE]

• 2.

Let $U_{2}$ be the set of set-valued tableaux $F\subseteq E$ each containing some tableau $f\in U$. Then,

[TABLE]

• 3.

Given a tableau $f\in U$ and $x\in X$, let $Y_{f}(x)$ be the set of $y\in Y$ such that $f(x)<y$ and moving the label on $x$ from $f(x)$ up to $y$ still yields a tableau in $U$. Then,

[TABLE]

We now consider the specific tableau complex such that the facets are the single-valued Rothe tableaux in ${\rm SRT}(w,\mathbf{f}_{0})$. To be consistent with the aforementioned notation, let $X=D(w)$ and $Y$ be the set of positive integers. Set $U={\rm SRT}(w,\mathbf{f}_{0})$ and

[TABLE]

We denote the above defined tableau complex by $\Delta(w)=\Delta_{E}(X\xrightarrow{U}Y)$, and call $\Delta(w)$ the Rothe tableau complex for $w$.

Using Theorem 1.1 and Theorem 3.1, we can now give a proof of Theorem 1.4.

Proof of Theorem 1.4. We define a partial ordering on $D(w)$ as follows. For two distinct squares $B$ and $B^{\prime}$ of $D(w)$, we use $B\rightarrow B^{\prime}$ to represent that either $B$ and $B^{\prime}$ are in the same row and $B$ lies to the right of $B^{\prime}$, or $B$ and $B^{\prime}$ are in the same column and $B$ lies above $B^{\prime}$. Define $B<B^{\prime}$ if there exists a sequence $(B=B_{1},B_{2},\ldots,B_{k}=B^{\prime})$ of squares of $D(w)$ such that

[TABLE]

For each square $B=(i,j)$ of $D(w)$, let $Y_{B}=\{1,2,\ldots,i\}$. Moreover, we set $\Psi$ to be the set of pairs $(B,B^{\prime})$ with $B<B^{\prime}$ such that $B$ and $B^{\prime}$ are in the same column of $D(w)$. Now we see that the tableaux satisfying the conditions $(a)$, $(b)$ and $(c)$ in Theorem 3.1 are exactly the single-valued Rothe tableaux in ${\rm SRT}(w,\mathbf{f}_{0})$. Recall that the set $U_{1}$ defined in Theorem 3.1 consists of the set-valued tableaux $F\subseteq E$ such that every tableau in $F$ lies in $U$. Clearly, $F\subseteq E$ is a set-valued tableau satisfying that every tableau contained in $F$ lies in $U$ if and only if $F$ is a set-valued Rothe tableau in ${\rm SVRT}(w,\mathbf{f}_{0})$. Thus we have $U_{1}={\rm SVRT}(w,\mathbf{f}_{0})$. Replacing $t_{x\arrownot\mapsto a}$ with $x=(i,j)\in D(w)$ by

[TABLE]

and then replacing $x_{t}$ by $1-x_{t}$ and $y_{t}$ by $\frac{1}{1-y_{t}}$, the $K$-polynomial $K(\mathds{k}[\Delta];\mathbf{t})$ in (3.1) becomes

[TABLE]

which agrees with the formula (1.2) in Theorem 1.1. Making the same substitutions in (3.2) and (3.3), we are led to ((1)) and ((2)) respectively. This completes the proof.

Acknowledgments. This work was supported by the 973 Project and the National Science Foundation of China.