This paper introduces a unified geometric framework for graph Schubert varieties and their variants, connecting them with back-stable Schubert polynomials and providing new formulas and interpretations.
Contribution
It establishes that the classes of these loci are represented by back-stable Schubert polynomials and extends the framework to symmetric and skew-symmetric cases with Pfaffian formulas.
Findings
01
Classes of graph Schubert varieties are represented by back-stable Schubert polynomials.
02
Derived new Pfaffian formulas for symmetric and skew-symmetric cases.
03
Provided geometric interpretations for involution Stanley symmetric functions.
Abstract
We consider the loci of invertible linear maps f:Cn→(Cn)∗ together with pairs of flags (E∙,F∙) in Cn such that the various restrictions f:Fj→Ei∗ have specified ranks. Identifying an invertible linear map with its graph viewed as a point in a Grassmannian, we show that the closures of these loci have cohomology classes represented by the back-stable Schubert polynomials of Lam, Lee, and Shimozono. As a special case, we recover the result of Knutson, Lam, and Speyer that Stanley symmetric functions represent the classes of graph Schubert varieties. We consider similar loci where f is restricted to be symmetric or skew-symmetric. Their classes are now given by back-stable versions of the polynomials introduced by Wyser and Yong to represent classes of orbit closures for the orthogonal and symplectic groups acting on…
Equations247
GXw=def{(G(f),E∙,F∙):rk(Fj↪CnfCn∗↠Ei∗)=rkw[i][j] for i,j∈[n]}
GXw=def{(G(f),E∙,F∙):rk(Fj↪CnfCn∗↠Ei∗)=rkw[i][j] for i,j∈[n]}
LG(2n)=def{U∈Gr(n,2n):(U,U)−=0}
LG(2n)=def{U∈Gr(n,2n):(U,U)−=0}
LGXy=def{(G(f),E∙):rk(Ej↪CnfCn∗↠Ei∗)=rky[i][j] for i,j∈[n]}
LGXy=def{(G(f),E∙):rk(Ej↪CnfCn∗↠Ei∗)=rky[i][j] for i,j∈[n]}
OGXz=def{(G(f),E∙):rk(Ej↪CnfCn∗↠Ei∗)=rkz[i][j] for i,j∈[n]}
OGXz=def{(G(f),E∙):rk(Ej↪CnfCn∗↠Ei∗)=rkz[i][j] for i,j∈[n]}
{G(f)∈GLG:rk(Ej↪CnfCn∗↠Ei∗)=rky[i][j] for i,j∈[n], f symmetric}
{G(f)∈GLG:rk(Ej↪CnfCn∗↠Ei∗)=rky[i][j] for i,j∈[n], f symmetric}
{G(f)∈GOG:rk(Ej↪CnfCn∗↠Ei∗)=rkz[i][j] for i,j∈[n], f skew-symmetric}.
{G(f)∈GOG:rk(Ej↪CnfCn∗↠Ei∗)=rkz[i][j] for i,j∈[n], f skew-symmetric}.
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Full text
Universal graph Schubert varieties
Brendan Pawlowski
Abstract.
We consider the loci of invertible linear maps f:Cn→(Cn)∗ together with pairs of flags (E∙,F∙) in Cn such that the various restrictions f:Fj→Ei∗ have specified ranks. Identifying an invertible linear map with its graph viewed as a point in a Grassmannian, we show that the closures of these loci have cohomology classes represented by the back-stable Schubert polynomials of Lam, Lee, and Shimozono. As a special case, we recover the result of Knutson, Lam, and Speyer that Stanley symmetric functions represent the classes of graph Schubert varieties.
We consider similar loci where f is restricted to be symmetric or skew-symmetric. Their classes are now given by back-stable versions of the polynomials introduced by Wyser and Yong to represent classes of orbit closures for the orthogonal and symplectic groups acting on the type A flag variety. Using degeneracy locus formulas of Kazarian and of Anderson and Fulton, we obtain new Pfaffian formulas for these polynomials in the vexillary case. We also give a geometric interpretation of the involution Stanley symmetric functions of Hamaker, Marberg, and the author: they represent classes of involution graph Schubert varieties in isotropic Grassmannians.
1. Introduction
Let G be a semisimple complex algebraic group. A pair (G,K) where K⊆G is a closed subgroup is a symmetric pair if K is the fixed point set of an involutive automorphism G→G. For parabolic subgroups P⊆G and Q⊆K, the product G/P×K/Q is a double flag variety for (G,K). In the case that Q=BK is a Borel subgroup of K, He, Nishiyama, Ochiai, and Oshima classified those (G,K) and P for which the K-action on G/P×K/BK has finitely many orbits [16].
We consider three such cases here:
(1)
G=SL(2n), K=S(GL(n)×GL(n))=def(GL(n)×GL(n))∩G, and G/P the Grassmannian Gr(n,2n) of n-planes in C2n;
2. (2)
G=Sp(2n), K=GL(n), and G/P the Lagrangian Grassmannian LG(2n), the subvariety of Gr(n,2n) consisting of those n-planes on which a fixed nondegenerate skew-symmetric form on C2n vanishes;
3. (3)
For n even, G=SO(2n), K=GL(n), and G/P the orthogonal Grassmannian OG(2n), one component of the subvariety of Gr(n,2n) consisting of those n-planes on which a fixed nondegenerate symmetric form on C2n vanishes.
In each case, we give descriptions in terms of rank conditions for those K-orbits on G/P×K/BK intersecting a certain open dense subset of G/P, and give formulas for the cohomology classes Poincaré dual to their closures.
We write Gr(n,2n) for the Grassmannian of n-planes in Cn⊕Cn∗. Given a linear map f:Cn→Cn∗, its graph G(f)=def{(v,f(v)):v∈Cn} is a point in Gr(n,2n). The map Hom(Cn,Cn∗)→Gr(n,2n), f↦G(f), is an open embedding; let GGr be the image of the invertible maps. Let Fl(n) denote the variety of complete flags in Cn, so K/BK=Fl(n)×Fl(n) in case (1) above. If M is a matrix, let M[i][j] denote its upper-left i×j corner. Identify a permutation w∈Sn with the permutation matrix having 1’s in positions (i,w(i)), and let [n]=def{1,2,…,n}.
Theorem 1.1** (Proposition 3.2 and Theorem 3.11).**
The S(GL(n)×GL(n))-orbits on GGr×Fl(n)×Fl(n) are the sets
[TABLE]
for all w∈Sn. The integral cohomology class [GXw] Poincaré dual to the Zariski closure GXw is represented by the back-stable double Schubert polynomialSw(x,−y).
Despite the name, a back-stable double Schubert polynomial is a formal power series, obtained as a limit of double Schubert polynomials; see Definition 2.5. Back-stable Schubert polynomials were introduced by Lam, Lee, and Shimozono in the context of Schubert classes in infinite flag varieties [22]; we do not know an explanation of Theorem 1.1 from this point of view.
The fiber of GXw in Fl(n) over a fixed (G(f),E∙)∈GGr×Fl(n) is a Schubert variety, and more generally the fiber of GXw in Fl(n)×Fl(n) over a fixed G(f)∈GGr is a double Schubert variety as described in [1] and [29]. On the other hand, the fiber of GXw in Gr(n,2n) over a fixed (E∙,F∙)∈Fl(n)×Fl(n) is a graph Schubert variety as defined by Knutson, Lam, and Speyer [21]; accordingly, we call GXw a universal graph Schubert variety. They showed that the class of a graph Schubert variety is represented by a Stanley symmetric function (see Definition 2.5). An appropriate specialization in Theorem 1.1 gives a new proof of this fact.
Definition 1.2**.**
A linear map f:Cn→Cn∗ is symmetric if f(v)(w)=f(w)(v) for v,w∈Cn, and skew-symmetric if f(v)(w)=−f(w)(v).
There are canonical (up to sign) nondegenerate symmetric and skew-symmetric forms (−,−)+ and (−,−)− on Cn⊕Cn∗, defined by ((v1,ω1),(v2,ω2))±=ω1(v2)±ω2(v1).
We take O(2n) and Sp(2n) to be the subgroups of GL(2n)=GL(Cn⊕Cn∗) preserving (−,−)+ and (−,−)−. The Lagrangian Grassmannian is the closed subvariety
[TABLE]
of Gr(n,2n); it is a homogeneous Sp(2n)-variety. The variety of points U∈Gr(n,2n) with (U,U)+=0 has two irreducible components; the component containing Cn⊕0 is the orthogonal GrassmannianOG(2n), and it is a homogeneous SO(2n)-variety.
Proposition 1.3**.**
A linear map f:Cn→Cn∗ is symmetric if and only if G(f)∈LG(2n), and skew-symmetric if and only if G(f)∈OG(2n).
Let GLG be the open set of graphs of invertible symmetric linear maps Cn→Cn∗ in LG(2n). Let GOG be the open set of graphs of invertible skew-symmetric linear maps Cn→Cn∗ in OG(2n), assuming n is even. In the next theorem, we view GL(n) as a subgroup of Sp(2n) and of SO(2n) via the embedding g↦(g−1)∗⊕g.
Theorem 1.4** (Proposition 5.2 and Theorems 5.9 and 5.11).**
The GL(n)-orbits on GLG×Fl(n) are the sets
[TABLE]
for y∈Sn an involution. For n even, the GL(n)-orbits on GOG×Fl(n) are the sets
[TABLE]
for z∈Sn a fixed-point-free involution. The cohomology classes [LGXy] and [OGXz] are represented by 2cyc(y)SyO and SzSp, respectively, where cyc(y) is the number of 2-cycles in y and SyO and SzSp are back-stable involution Schubert polynomials.
Let In be the set of involutions in Sn, and In\textscfpf⊆In the subset of fixed-point-free involutions. The involution Schubert polynomialsSyO and SzSp were introduced by Wyser and Yong [32], who showed that the polynomials {2cyc(y)SyO:y∈In} represent the classes of the O(n)-orbit closures on Fl(n), and the polynomials {SzSp:z∈In\textscfpf} represent the classes of the Sp(n)-orbit closures on Fl(n). The back-stable involution Schubert polynomials SyO and SzSp are obtained from SyO and SzSp by a limiting process; see Definition 4.4.
The connection to our situation is as follows. Fix an invertible symmetric map f:Cn→Cn∗, and let O(n)⊆GL(n) be the subgroup preserving the symmetric form (v,w)↦f(v)(w). Then as y ranges over In, the fibers in Fl(n) of the various LGXy over G(f) are exactly the O(n)-orbits on Fl(n). Similarly, if f is skew-symmetric, then for z∈In\textscfpf, the fibers of the various OGXz over G(f) are the Sp(n)-orbits on Fl(n). This reversal explains why we write representatives for [LGXy] and [OGXz] as 2cyc(y)SyO and SzSp respectively, which at first may appear backwards.
Definition 1.5**.**
The involution graph Schubert varietyLGXy(E∙)⊆LG(2n) associated to y∈In is the fiber of LGXy over a fixed flag E∙∈Fl(n). The fixed-point-free involution graph Schubert variety associated to z∈In\textscfpf is the fiber of OGXy(E∙)⊆OG(2n) over a fixed flag E∙∈Fl(n). Explicitly, LGXy(E∙) is the closure of
[TABLE]
and OGXz(E∙) is the closure of
[TABLE]
A corollary of Theorem 1.4 is that [LGXy(E∙)]∈H∗(LG(2n),Z) is represented by the involution Stanley symmetric function2cyc(y)FyO introduced in [13], and that [OGXz(E∙)]∈H∗(OG(2n),Z) is represented by the fixed-point-free involution Stanley symmetric function FzSp. It was shown algebraically and combinatorially in [12, 15] that 2cyc(y)FyO and FzSp are positive integer combinations of Schur’s Q-functions and P-functions, respectively. By work of Pragacz [26], this positivity means that these symmetric functions represents cohomology classes of subvarieties in LG(2n) and OG(2n), and part of the motivation for the current work was to find such subvarieties.
Working in the other direction, Theorem 1.4 together with Pragacz’s results provides a new proof that 2cyc(y)FyO is Schur Q positive and FzSp is Schur P positive (Corollary 5.13). Their Schur Q and P expansions can be developed by explicit recurrences, found in [12, 15], that are analogous to the “transition recurrences” of Lascoux and Schützenberger [24]. As a special case, every product of Schur Q or P functions can be written as some 2cyc(y)FyO or FzSp, and so such transition recurrences give new Littlewood-Richardson rules for these families of symmetric functions, or equivalently for the Schubert bases of the integral cohomology of LG(2n) and OG(2n). Our hope is that the geometric perspective developed here will be helpful in finding similar rules in other cohomology theories—for instance, in the currently open problem of describing the Schubert structure constants of the K-theory ring K0(LG(2n)).
Other results on involution Stanley symmetric functions from [12, 15] are also clarified by the geometric perspective. For instance, “I-Grassmannian” and “fpf-I-Grassmannian” involutions were singled out on combinatorial grounds as base cases for the transition recurrences mentioned above, filling the role played by Grassmannian permutations in the classical case. The special role of these involutions is clear from the geometry: the involution graph Schubert varieties they index are simply Schubert varieties in LG(2n) and OG(2n).
Pfaffian formulas for involution Schubert polynomials and Stanley symmetric functions were given in [12, 15], in the case of I-Grassmannian and fpf-I-Grassmannian involutions. We improve on these formulas by adding a missing base case and generalizing them to vexillary involutions. This is done by realizing LGXz(E∙) and OGXz(E∙) as type C and type D Grassmannian degeneracy loci in the sense of Kazarian [19] and Anderson and Fulton [3]. The interpretation of GXw, LGXz, and OGXz as certain degeneracy loci when w and z are vexillary is a key element in our proofs of Theorems 1.1 and 1.4 as well.
1.1. Outline
In §2, we recall basic facts about cohomology rings of flag varieties, degeneracy loci, and Schubert varieties, and prove some combinatorial lemmas on vexillary permutations. In §3, we characterize the universal graph Schubert varieties GXw as closures of certain GL(n)×GL(n)-orbits on Gr(n,2n)×Fl(n)×Fl(n), and show that their classes [GXw] are represented by double back-stable Schubert polynomials, proving Theorem 1.1. Sections 4 and 5 recapitulate this story for involution graph Schubert varieties: in particular, in §5 we prove Theorem 1.4 and give Pfaffian formulas for SzO when z is vexillary. The notion of “vexillary” and accompanying Pfaffian formulas are more delicate in the fixed-point-free case, and the necessary combinatorics is developed in §6, where the particular case of I-Grassmannian involutions is also investigated. In Section 7, we combine the results of §5 and formulas of Ivanov to express SzO in terms of shifted tableaux when z is vexillary.
Acknowledgements
I am grateful to Bill Fulton for teaching a course on degeneracy loci exactly when I needed to learn about them; I also thank Zach Hamaker, Thomas Lam, and Eric Marberg for helpful conversations, and Mark Shimozono for asking questions that eventually motivated some of the main results here.
2. Preliminaries
2.1. Cohomology
Let X be a smooth complex variety. Throughout we write H∗(X) for the integral singular cohomology ring H∗(X,Z). Suppose π:E↠X is a complex vector bundle. For x∈X, we write Ex for the fiber π−1(x), a complex vector space. The total Chern classc(E) of E is an element of H∗(X), not necessarily homogeneous, which is zero outside degrees 0,2,4,…,2rk(E). The dth Chern classcd(E) or c(E)d is the degree 2d component of c(E). The properties of Chern classes we need are:
(a)
c0(E)=1.
2. (b)
c(F)=c(E)c(G) when 0→E→F→G→0 is a short exact sequence.
3. (c)
cd(E∗)=(−1)dcd(E).
4. (d)
If E is trivial, then c(E)=1.
Since H∗(X) is zero in large enough degree, c0(E)=1 implies that c(E) is a unit. Part (b) above then says that c(E⊕F)=c(E)c(F) and c(E/F)=c(E)/c(F).
Define alphabets
[TABLE]
Let Fl(n) be the set of complete flags E∙ in Cn, i.e. chains E1⊆E2⊆⋯⊆En=Cn where Ei is an i-dimensional linear subspace. For each i there is a tautological vector bundleEi↠Fl(n), whose fiber (Ei)E∙ over a point E∙∈Fl(n) is the subspace Ei. Borel showed that the map
[TABLE]
sending xi↦c1((Ei/Ei−1)∗) for i=1,…,n is a well-defined isomorphism, where ed is the degree d elementary symmetric function [7]. We write E∙ and F∙ for the tautological flags of bundles over the two factors of Fl(n)×Fl(n), and let yi=c1((Fi/Fi−1)∗), so that members of H∗(Fl(n)×Fl(n))≃H∗(Fl(n))⊗H∗(Fl(n)) can be represented by polynomials in x+ and y+.
The projection Fl(n)→Gr(k,n) sending E∙↦Ek induces as its pullback an inclusion H∗(Gr(k,n))↪H∗(Fl(n)) whose image is the subring of (Sk×Sn−k)-invariants in Z[x1,…,xn]/(e1(x1⋯n),…,en(x1⋯n)). This subring is isomorphic to Λ/(ek+1,ek+2,…), where Λ is the ring of symmetric functions over Z. With this identification, the dual tautological bundle G∗↠Gr(k,n) has Chern classes e0,e1,…,ek.
However, in our setting it seems more natural to consider the maps Fl(n)×Fl(n)→Gr(n,2n) sending (E∙,F∙)↦Fi⊕(Cn/Ei)∗, for each i∈[n]. Under this map, the dual tautological bundle G∗↠Gr(n,2n) pulls back to Fi∗⊕(Cn/Ei), so the induced map H∗(Gr(n,2n))→H∗(Fl(n))⊗H∗(Fl(n)) sends
[TABLE]
hence cd(G∗)↦∑a+b=dha(x1⋯i)eb(y1⋯i), where ha is the degree a complete homogeneous symmetric function. This suggests the next definition.
Definition 2.1**.**
Let Δ:Λ→Λ⊗Λ be the coproduct on symmetric functions defined by Δ(ed)=∑a+b=dea⊗eb, and ω:Λ→Λ the ring involution sending ed to hd. The ring Λsuper of supersymmetric functions is the image of (ω⊗id)∘Δ in Λ⊗Λ.
We view ∑ifi⊗gi∈Λsuper as the formal power series ∑ifi(x−)gi(y−). For f∈Λ, write f(x∖y)=def((ω⊗id)∘Δ)(f); so, for instance, ed(x∖y)=∑a+b=dha(x)eb(y). As suggested by (2), we identify H∗(Gr(k,n)) with a quotient of Λsuper by sending ed(x∖y)↦cd(G∗). Equivalently, ∑dhd(x∖y) represents 1/c(G).
Remark 2.2**.**
It is more common to define supersymmetric functions as the image of one of the maps f↦(1⊗ω)(Δ(f)) or f↦(1⊗(−1)deg(f)ω)(Δ(f)), and to write f(x/y) for the image of f. We have used the notation f(x∖y) instead to reflect our different convention.
Since Grassmannians and flag varieties have no odd-dimensional cohomology, the Künneth theorem and universal coefficient theorem imply that the natural map
[TABLE]
is an isomorphism. Thus, classes in H∗(Gr(n,2n)×Fl(n)×Fl(n)) can be represented by members of Λsuper⊗Z[x+]⊗Z[y+], which we view as formal power series in x∪y.
2.2. Schubert polynomials and Stanley symmetric functions
Definition 2.3**.**
A compatible sequence for a word a=a1⋯aℓ is a weakly increasing word i1≤⋯≤iℓ with entries in Z∖{0} such that for each j, (1) ij≤aj, and (2) if aj<aj+1 then ij<ij+1. Let C(a) be the set of compatible sequences for a.
Our definition of compatible sequence is slightly different from usual (e.g. [5]), in that i is typically required to have positive entries.
Let R(w) be the set of reduced words of a permutation w: the minimal-length words a1⋯aℓ such that sa1⋯saℓ=w, where si is the transposition (i,i+1). If a is a word and m∈Z, we write m≤a to mean that m≤ai for each i.
Example 2.4**.**
We use bold to distinguish reduced words and compatible sequences from permutations. For instance, R(2143)={13,31} and
[TABLE]
If f is a formal power series in variables x, we write (for instance) f\bigr{\rvert}_{x_{-}\to 0} to indicate the result of setting the variables in x− to zero.
Definition 2.5**.**
The back-stable Schubert polynomial [22] of w∈Sn is
Despite its name, Sw is not a polynomial but a formal power series in x. In accordance with their names, Sw is a polynomial in x+, and Fw is a symmetric function in x− (this symmetry is not obvious from our definition).
where we view the elementary and homogeneous symmetric functions ed and hd as formal power series in variables x−. Setting xi to [math] for i<0 gives S2143=x12+x1x2+x1x3, and setting xi to [math] for i>0 gives F2143=e2+h2.
Remark 2.7**.**
Back stable Schubert polynomials can be defined in terms of ordinary Schubert polynomials. For v∈Sm and w∈Sn, let v×w∈Sm×n be the permutation
[TABLE]
Let 1m∈Sm denote the identity permutation. Then Sw=limm→∞S1m×w(x−m⋯n). The next definition uses a similar approach.
For u,v,w∈Sn, write uv≐w to mean that uv=w and ℓ(w)=ℓ(u)+ℓ(v). Here, ℓ(w) is the number of inversions of w, or equivalently the length of any a∈R(w).
The back-stable double Schubert polynomial of w is
[TABLE]
The divided difference operator∂i sends f∈R[x1,x2,…] to
[TABLE]
where R is a commutative ring and si=(i,i+1) acts on R[x1,x2,…] by interchanging xi and xi+1. If there are x-variables and y-variables, ∂i will always act on the x-variables and treat the y-variables as scalars. That is, we take the action of ∂i on R[x1,x2,…,y1,y2,…] to be the action of ∂i on S[x1,x2,…] where S=R[y1,y2,…].
The back-stable double Schubert polynomials Sw(x;y) satisfy the recurrence
[TABLE]
Let w0∈Sn be the reverse permutation n(n−1)⋯21. Any w∈Sn can be reached starting from w0 via a sequence of transformations v⇝vsi where ℓ(vsi)<ℓ(v), so Proposition 2.9 inductively determines every Sw(x;y) once Sw0(x;y) is known.
Definition 2.10**.**
The Schubert varietyXw associated to w∈Sn with respect to a fixed flag E∙′⊆Cn is the closure of the Schubert cell
[TABLE]
Suppose we represent a flag E∙⊆Cn by a matrix A so that Ei is the span of the first i rows of A for each i. Taking Ei′ to be the span of the standard basis vectors en,…,en−i+1, the Schubert cell Xw consists of flags with matrix A such that rkA[i][j]=rkw[i][j] for i,j∈[n]. To obtain the closed Schubert variety Xw, replace the equalities rkA[i][j]=rkw[i][j] with inequalities rkA[i][j]≤rkw[i][j]. Schubert varieties are irreducible, and codimXw=ℓ(w).
Lascoux and Schützenberger introduced the Schubert polynomial Sw as a representative for the class [Xw]∈H∗(Fl(n)) Poincaré dual to Xw [23]. The Schubert cells {Xw:w∈Sn} are the cells of a CW decomposition of Fl(n), and consequently {Sw:w∈Sn} is a Z-basis of Z[x1,…,xn]/(e1(x1⋯n),…,en(x1⋯n))≃H∗(Fl(n)).
The Schubert polynomials satisfy an important stability property: Sw×1=Sw for any w∈Sn. Viewing Sn as a subgroup of Sn+1 via the embedding w↦w×1, let S∞=def⋃n≥0Sn. The stability property of Schubert polynomials means it is well-defined to write Sw for w∈S∞. Moreover, {Sw:w∈S∞} forms a basis of Z[x1,x2,…] [11, Ch. 10].
Lemma 2.11**.**
Let αi∈H∗(Fl(ni)) be a sequence of classes where (ni) is a sequence tending toward ∞. There is at most one polynomial f which represents every class αi.
Proof.
Suppose f represents every αi. Since {Sw:w∈S∞} is a basis of Z[x1,x2,…], we can write f=∑w∈SnicwSw for some sufficiently large i. Since f represents αi, we then have αi=∑w∈Snicw[Xw]. The classes {[Xw]:w∈Sni} are linearly independent, so the coefficients cw can be determined from αi.
∎
2.3. Vexillary permutations
Definition 2.12**.**
The Rothe diagram of w∈Sn is the set
[TABLE]
Definition 2.13**.**
The essential set of a set D⊆N×N is
[TABLE]
That is, Ess(D) is the set of southeast corners of connected components of D, viewing two elements of N×N as connected if they are vertically or horizontally adjacent.
Definition 2.14**.**
A permutation w∈Sn is vexillary if it avoids the pattern 2143, i.e. there do not exist i<j<k<l in [n] with w(j)<w(i)<w(l)<w(k).
Let
↗
≤
be the partial order on N×N increasing from southwest to northeast, meaning that (i,j)\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-2.65mm}\hbox{\leq}}(i^{\prime},j^{\prime}) if and only if i≥i′ and j≤j′. If i∈N and S⊆N, we write DiS(w) for {j∈S:(i,j)∈D(w)}, or simply DiS when w is understood.
The equivalences of (a) with parts (b) and (c) in the next lemma are due to Fulton [10] and Wachs [30], respectively.
Lemma 2.15**.**
The following are equivalent:
(a)
w∈Sn* is vexillary.*
2. (b)
Ess(D(w))* is a chain under *
* *
↗
≤
.
3. (c)
The sets DiN(w) for i∈N are totally ordered under inclusion.
Example 2.16**.**
Let w be the vexillary permutation 35142. Each × in the following diagram is a point (i,w(i)) in matrix coordinates, with the points of the Rothe diagram D(w) marked by squares: they are the points directly left of a × and directly above a ×. Elements of Ess(D(w)) are marked by black squares. All points are drawn in matrix coordinates, with (1,1) at the upper left:
[TABLE]
By contrast, the subsequence 3154 of v=31524 is a 2143 pattern, the diagram
[TABLE]
has an essential set element (1,2) strictly northwest of the essential set element (3,4), and the sets D1N(v)={1,2} and D3N(v)={2,4} are incomparable under containment.
Definition 2.17**.**
The code of w∈Sn is the list c(w)=(c1(w),…,cn(w)) where ci(w)=#{j>i:w(j)<w(i)}=∣DiN(w)∣. The shapesh(w) of w is the transpose of the partition obtained by sorting c(w) and ignoring [math]’s.
We note that the shape of w is more commonly defined as sh(w)t.
Example 2.18**.**
The code of 35142 is (2,3,0,1,0), and the shape is (3,2,1)t=(3,2,1).
In the remainder of this subsection, we prove some lemmas which we will need to extract the rank conditions defining certain degeneracy loci, described in the next subsection, from the combinatorics of Rothe diagrams and essential sets.
Lemma 2.19**.**
Suppose w is vexillary. Write \operatorname{Ess}(D(w))=\{(i_{1},j_{1})\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-3.05mm}\hbox{<}}\cdots\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-3.05mm}\hbox{<}}(i_{s},j_{s})\}, and let kp=jp−rkw[ip][jp] for p∈[s]. Then {k1,…,ks}={c1(w),…,cn(w)}∖{0}.
Proof.
We will repeatedly use the fact that ci(w) is the number of elements of row i of D(w), and that kp is the number of elements in row ip and columns [jp]. That is, ci(w)=∣DiN(w)∣ and kp=∣Dip[jp](w)∣.
Given i∈[n] with ci(w)>0, let j be maximal such that (i,j)∈D(w). Let i′≥i be maximal such that (i,j),(i+1,j),…,(i′,j)∈D(w). We proceed by proving a series of claims.
(a)
(i′,j)∈Ess(D(w)): The maximality of j means that w−1(j+1)≤i≤i′, so that (i′,j+1)∈/D(w), and likewise the maximality of i′ means that (i′+1,j)∈/D(w).
2. (b)
ci(w)=∣Di[j]∣: By definition ci(w)=∣DiN∣, and DiN=Di[j] by the choice of j.
3. (c)
ci(w)∈{k1,…,ks}: By (a), i′=ip for some p. The fact that D(w) contains (r,j) for i≤r≤i′ implies that w(r)>j for all such r, so Di[j]=Dr[j] for such r. In particular, taking r=i′ and using (b), ci(w)=∣Di[j]∣=∣Di′[j]∣=kp.
Conversely, take p∈[s]. We want to find i such that kp=ci(w). Let S={j>jp:(ip,j)∈D(w)}. If S=∅, then kp=cip(w) and we are done. Otherwise, let i<ip be maximal so that jp<w(i)<min(S); such an i exists because (ip,jp)∈Ess(D(w)) implies that w−1(jp+1) satisfies the conditions demanded of i. Now:
(a)
∣Di[jp]∣=ci(w): Suppose not, so there is some j>jp with (i,j)∈D(w). Then w−1(j) satisfies the conditions used to choose i, namely: jp<w(w−1(j))=j; j<w(i)<min(S) because (i,j)∈D(w); and w−1(j)<ip because otherwise j∈S, contradicting j<min(S). However, w−1(j)>i because (i,j)∈D(w), so this would contradict the maximality of i.
2. (b)
If i≤r≤ip than w(r)>jp: The choice of i says that w(i)>jp, and w(ip)>jp because (ip,jp)∈D(w), so assume i<r<ip. Since r<ip and (ip,jp)∈D(w), we have w(r)=jp, so suppose for the sake of contradiction that w(r)<jp. Because (ip,min(S))∈D(w) we have ip<w−1(min(S)) and min(S)<w(ip), and now w contains a 2143 pattern: i<r<ip<w−1(min(S)) and w(r)<jp<w(i)<min(S)<w(ip). This contradicts the assumption that w is vexillary.
3. (c)
kp∈{c1(w),…,cn(w)}: Part (b) implies that Di[jp]=Dr[jp] for i≤r≤ip. In particular, taking r=ip and using (a) gives kp=∣Dip[jp]∣=∣Di[jp]∣=ci(w). ∎
Lemma 2.20**.**
Suppose w is vexillary. Write \operatorname{Ess}(D(w))=\{(i_{1},j_{1})\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-3.05mm}\hbox{<}}\cdots\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-3.05mm}\hbox{<}}(i_{s},j_{s})\}, and let kp=jp−rkw[ip][jp] for p∈[s], and k0=0. If kp−1<k≤kp, then sh(w)k=sh(w)kp=ip−rkw[ip][jp].
Proof.
Again we proceed by proving a series of claims, whose aim is to establish that {i:ci(w)≥kp}={i≤ip:(i,jp)∈D(w)}. Note that the size of the lefthand set is sh(w)kp, while the size of the righthand set is ip−rkw[ip][jp].
(a)
If (i,jp)∈D(w) and i≤ip, then ci(w)≥kp: It is not hard to see that D(w) is closed under taking northwest corners in the sense that if (a,b)\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-2.65mm}\hbox{\leq}}(a^{\prime},b^{\prime}) are in D(w), then (a′,b)∈D(w) also. In particular, if j∈Dip[jp], then (i_{p},j)\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-2.65mm}\hbox{\leq}}(i,j_{p}), so D(w) contains (i,j). Therefore kp=∣Dip[jp]∣≤∣Di[jp]∣≤ci(w).
2. (b)
If (i,jp)∈/D(w) and i≤ip, then ci(w)<kp: These assumptions force w(i)<jp, so ∣Di[jp]∣=∣DiN∣=ci(w). Given that the sets DiN for i∈N are totally ordered under containment by Lemma 2.15, (i,jp)∈/D(w) and (ip,jp)∈D(w) means Di[jp]⊊Dip[jp], hence ci(w)=∣DiN∣=∣Di[jp]∣<∣Dip[jp]∣=kp.
3. (c)
If i>ip, then ci(w)<kp: First let us see that ci(w)=∣Di[jp]∣. If not, there is some (i,j)∈D(w) with j>jp. Then (ip,jp) is strictly north and west of (i,j), and hence of any essential set element in the same connected component of D(w) as (i,j). By Lemma 2.15, this contradicts the assumption that w is vexillary.
Now, since (ip,jp)∈Ess(D(w)), we have w(ip+1)≤jp, so D(w) contains (ip,w(ip+1)) but not (i,w(ip+1)). As in (b), this implies Di[jp]⊊Dip[jp], and hence ci(w)=∣Di[jp]∣<∣Dip[jp]∣=kp.
Parts (a), (b), and (c) together prove that {i:ci(w)≥kp}={i≤ip:(i,jp)∈D(w)}, hence sh(w)kp=ip−rkw[ip][jp] by comparing cardinalities. The distinct parts of sh(w)t are {k1,…,ks} by Lemma 2.19, so if kp−1<k≤kp then sh(w)k=sh(w)kp.
∎
We conclude with a technical lemma to be used in §4.5.
Lemma 2.21**.**
Suppose w is vexillary. Write \operatorname{Ess}(D(w))=\{(i_{1},j_{1})\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-3.05mm}\hbox{<}}\cdots\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-3.05mm}\hbox{<}}(i_{s},j_{s})\}, and let kp=jp−rkw[ip][jp] for p∈[s], and k0=0. If (ip,jp−1)∈/D(w) for some p>1, then kp=kp−1+1.
Proof.
We consider two cases:
•
Suppose jp−1<jp. By the northwest closure property of D(w), the cell (ip,jp−1) is in D(w) given that (ip−1,jp−1) and (ip,jp) are. If (i,jp−1)∈/D(w) for some ip<i<ip−1, then the connected component of (ip,jp−1) has a southeast corner (i′,j′)∈Ess(D(w)) strictly above row ip−1, and strictly left of column jp given that (ip,jp−1)∈/D(w). But then (i_{p-1},j_{p-1})\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-3.05mm}\hbox{<}}(i^{\prime},j^{\prime})\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-3.05mm}\hbox{<}}(i_{p},j_{p}), which is impossible. We conclude that D(w) contains (i,jp−1) for all ip≤i≤ip−1. This implies that w(i)>jp−1 for all such i, so Dip−1[jp−1]=Dip[jp−1].
A similar argument shows that D(w) does not contain (ip,j) whenever jp−1<j<jp: otherwise there would be an essential set cell strictly right of column jp−1, but strictly left of column jp given that (ip,jp−1)∈/D(w). So, Dip[jp]=Dip[jp−1]∪{jp}. We conclude that
[TABLE]
•
Suppose jp−1=jp, so ip−1<ip. Since D(w) contains (ip,jp) but not (ip,jp−1) or (ip+1,jp), we have w−1(jp−1)<ip and w(ip+1)<jp. This inequalities together actually imply w−1(jp−1)<ip and w(ip+1)<jp−1. Also, note that ip+1<ip−1 since D(w) contains (ip−1,jp) but not (ip+1,jp). But now w−1(jp−1)<ip+1<ip−1<w−1(jp) and w(ip+1)<jp−1<jp<w(ip−1), so w contains a 2143 pattern, a contradiction: this case cannot occur. ∎
2.4. Degeneracy locus formulas
Let E1↪⋯↪En and Fn↠⋯↠F1 be sequences of vector bundles over a smooth variety X, where rkEi=rkFi=i. Let f:En→Fn be a bundle map, so we are given a linear map fx:(En)x→(Fn)x for each x∈X. The corresponding degeneracy locusΩw labeled by w∈Sn is the closure of the locus Ωw of points in X over which
[TABLE]
for i,j∈[n]. That is, \Omega_{w}=\{x\in X:\text{\operatorname{rk}(f_{x}:(\mathcal{E}{i}){x}\to(\mathcal{F}{j}){x})=\operatorname{rk}w_{[i][j]}fori,j\in[n]}\}. Fulton gave a formula for the class of Ωw when things are suitably generic.
If X is smooth and Ωw has codimension ℓ(w), then [Ωw]=Sw(x′;y′), where xi′=c1((Ei/Ei−1)∗) and yi′=c1(Fi∗/Fi−1∗).
Example 2.23**.**
Take E∙ to be the tautological flag of bundles over Fl(n) as before. Fix a flag E∙′∈Fl(n) and let Fi be the trivial bundle Cn/En−i′ over Fl(n); we adopt the common abuse of simply writing V for the trivial bundle V×Fl(n)→Fl(n). Let f:Cn=En→Fn=Cn be the identity map. Then Ωw=Xw. Under our conventions from §2.1, xi′=xi and yi′=0, so Theorem 2.22 implies that [Xw] is represented by the (single) Schubert polynomial Sw(x)=Sw(x;0).
Lemma 2.24**.**
The degeneracy locus Ωw is the closure of the locus where
[TABLE]
for (i,j)∈Ess(D(w)).
This lemma is more or less equivalent to results of Fulton in [10], but is not quite stated in the same way, so we give a proof for completeness and because we will need a similar result in a different setting later.
Proof.
Define Ωw=,ess, Ωw≤,ess, Ωw=, and Ωw≤ as the four loci in X over which the ranks rk(Ei↪EnfFn↠Fj) are either equal to or at most the ranks rkw[i][j], either for (i,j)∈Ess(D(w)) or for all (i,j)∈[n]×[n]. Define Mw=,ess, Mw≤,ess, Mw=, and Mw≤ as the sets of g∈Hom(Cn,Cn) where the ranks rk(Ci↪CngCn↠Cn/Cn−j) are similarly constrained. We want to show that Ωw=,ess=Ωw=.
First we prove the lemma in the case that Ei=Ci and Fi=Cn/Cn−i are trivial. Specifying a bundle map f:En→Fn is then equivalent to specifying a map ϕ:X→Hom(Cn,Cn), and (for instance)
[TABLE]
It therefore suffices to show that Mw=,ess=Mw=. It is clear from the definitions that Mw≤,ess⊇Mw=,ess⊇Mw=. Fulton showed that Mw==M≤=M≤,ess [10, Proposition 3.3 and Lemma 3.10], so we must also have Mw=,ess=Mw=.
Now suppose E∙ and F∙ are trivial over some open set U⊆X. Replacing X by U, the relevant loci are Ωw=,ess∩U and Ωw=∩U, and their closures (in U) are Ωw=,ess∩U and Ωw=∩U. The previous paragraph implies Ωw=,ess∩U=Ωw=∩U. Since X is covered by open sets U over which E∙ and F∙ are trivial, we conclude that Ωw=,ess=Ωw=.
∎
Suppose V is a vector bundle over X with a rank r subbundle G and a flag of subbundles Hμ1⊆⋯⊆Hμs⊆V, where rkHi=rkV−i, so μ1≥⋯≥μs. Also fix a sequence 0=k0<k1<⋯<ks of integers. The Grassmannian degeneracy locusΩGr with respect to this data is the closure of the locus in X over which dim(G∩Hμi)=ki for each i, or equivalently rk(G↪V↠V/Hμi)=r−ki. For each k∈[ks], let p be such that kp−1<k≤kp, and define λk=μp−r+kp and c(k)=c(V)/(c(G)c(Hμp)).
Theorem 2.25**.**
If the sequence λ just defined is a partition and ΩGr has codimension ∣λ∣=def∑iλi, then [ΩGr]=det(c(k)λk+t−k)k,t∈[ℓ(λ)].
This formula is a modest generalization of a formula of Kempf and Laksov [20]. Alternatively, it can be deduced from Theorem 2.22 as follows. Let n=rkV and define w∈Sn by wk=λr−k+1+k for k≤r and {wr+1<⋯<wn}=[n]∖{w1,…,wr}. Then Ess(D(w))={(r,μp):p∈[s]} and rkw[r][μp]=r−kp for i∈[s]. Without loss of generality one can assume that the partial flags H∙ and G can be completed to complete flags in V, so ΩGr=Ωw by Lemma 2.24. Theorem 2.25 then follows from a similar determinantal formula for Sw(x;y); see [10, Proposition 9.18] or [30].
3. GL(n)×GL(n)-orbits on Gr(n,2n)×Fl(n)×Fl(n)
3.1. Description of orbits
Let G_{\operatorname{Gr}}\overset{\text{def}}{=}\{\mathsf{G}(f)\mid\text{f:\mathbb{C}^{n}\to{\mathbb{C}^{n}}^{*} linear and invertible}\}. Write elements of Gr(n,2n)=Gr(n,Cn⊕Cn∗) as row spans of n×2n matrices whose first n columns are coordinates on Cn and last n columns are coordinates on Cn∗. The complement of GGr is the closed locus where the Plücker coordinates in columns [n] and in columns [n+1,2n] both vanish, so GGr is Zariski-open. For w∈Sn, define GXw⊆GGr×Fl(n)×Fl(n) to be
[TABLE]
The expression FfE∗ here refers to the composition F↪CnfCn∗↠E∗ for subspaces E,F⊆Cn and a linear map f:Cn→Cn∗. In this subsection we show that the GXw are the GL(n)×GL(n)-orbits on GGr×Fl(n)×Fl(n), or equivalently the S(GL(n)×GL(n))-orbits, as discussed in the introduction.
Suppose X=∏j∈JXj is a Cartesian product of sets, Y=∏i∈IXi for some subset I⊆J, and π:X→Y is the projection. For U⊆X and y∈Y, we write U(y) for the projection of the fiber π−1({y})∩U onto ∏j∈J∖IXj.
Example 3.1**.**
Let e1,…,en be the standard basis of Cn, with dual basis e1∗,…,en∗, and let f:Cn→Cn∗ be linear with ei↦ei∗ for i∈[n]. Fix a flag E∙0=F∙0∈Fl(n) defined by Ei0=Fi0=⟨e1,…,ei⟩ for i∈[n], and write F∙=rowspan∙N to mean that Fi is the span of rows 1,…,i of a matrix N. With respect to the projection from GXw onto the first two factors of GGr×Fl(n)×Fl(n), the fiber GXw(G(f),E∙0) is
[TABLE]
This is the Schubert cell Xw−1 with respect to the flag E∙0. Consider on the other hand the projection from GGr×Fl(n)×Fl(n) onto its first and third factors. Then since rk(FjfEi∗)=rk(Eif∗Fj∗), the fiber GXw(G(f),F∙0) is
[TABLE]
This is the Schubert cell Xw with respect to F∙0. We will continue the convention used here of using E∙ and F∙ for the two coordinates of Fl(n)×Fl(n).
Let Bn+ and Bn− be the groups of invertible upper and lower triangular matrices inside GL(n). As in Example 3.1, fix E∙0∈Fl(n) defined by Ei0=span{e1,…,ei} for i∈[n]. The action of (g1,g2)∈K=defGL(n)×GL(n) on U∈Gr(n,2n) is given by (g1,g2)⋅U=(g2⊕(g1−1)∗)U.
Proposition 3.2**.**
The sets GXw for w∈Sn are the K-orbits on GGr×Fl(n)×Fl(n).
Proof.
The group K acts transitively on Fl(n)×Fl(n) with the stabilizer of (E∙,E∙) being Bn+×Bn+. So, the K-orbits on GGr×Fl(n)×Fl(n) are the sets
[TABLE]
where (gBn+,hBn+) ranges over GL(n)/Bn+×GL(n)/Bn+ and O ranges over the Bn+×Bn+-orbits on GGr. We claim that the latter orbits are the sets GXw(E∙0,E∙0), which will imply the proposition given that it is straightforward to check that
[TABLE]
Let (g1,g2)∈K act on M∈GL(n) by (g1,g2)⋅M=(g1t)−1Mg2−1. If G(f)∈GGr, then (g1,g2)⋅G(f)=G((g1−1)∗∘f∘g2−1). Thus, the isomorphism GGr→GL(n) taking G(f) to the matrix of f with respect to the ordered bases e1,…,en and e1∗,…,en∗ is GL(n)-equivariant, so sends Bn+×Bn+-orbits to Bn+×Bn+-orbits. This isomorphism also sends GXw(E∙0,E∙0) to the set O_{w}=\{M\in\operatorname{GL}(n):\text{\operatorname{rk}M_{[i][j]}=\operatorname{rk}w_{[i][j]}fori,j\in[n]}\}. The sets Ow are the Bn+×Bn+-orbits on GL(n) by [10, Lemma 3.1], proving the claim.
∎
It will sometimes be convenient to know that GXw or a similar variety is a fiber bundle over some subproduct of Gr(n,2n)×Fl(n)×Fl(n). In each case this will follow from the next lemma. If a group G acts on a set X, let Gx be the stabilizer of x∈X and Gx the orbit of x.
Lemma 3.3**.**
Let G be a Lie group acting on spaces X and Y, and hence diagonally on X×Y. Suppose the action on X is transitive. If S⊆X×Y is G-stable, the projection p:S→X onto the first coordinate is a fiber bundle.
Proof.
Fix x∈X. Since Gx⊆G is a closed subgroup, the projection q:G↠G/Gx≃X is a fiber bundle [8, Theorem 4.3]. Let ϕ:q−1(U)→U×Gx be a trivialization of q over some neighborhood U of x. Define ψ:p−1(U)→U×p−1(x) by the formula (gx,y)↦(gx,x,ϕ−1(gx,1)−1y), using the transitivity of the G-action on X. It is not quite obvious that ψ(gx,y) actually lies in U×p−1(x)⊆U×S when (gx,y)∈S: this holds because ϕ−1(gx,1)x=q(ϕ−1(gx,1))=gx, so that (x,ϕ−1(gx,1)−1y)=ϕ−1(gx,1)−1⋅(gx,y) is still in S by the G-stability of S. The map ψ has (gx,x,y)↦(gx,ϕ−1(gx,1)y) as an inverse, and trivializes p over U.
∎
Corollary 3.4**.**
The projection from GXw to any proper subproduct of GGr×Fl(n)×Fl(n) is a fiber bundle.
Lemma 3.5**.**
(a)
GXw* is irreducible of codimension ℓ(w).*
2. (b)
GXw* is the closure in Gr(n,2n)×Fl(n)×Fl(n) of*
[TABLE]
3. (c)
GXw(G(f))=GXw(G(f))* for any G(f)∈GGr.*
4. (d)
The intersection GXw(G(f))=GXw∩({G(f)}×Fl(n)×Fl(n)) is transverse.
Proof.
(a)
Given that GXw is a GL(n)×GL(n)-orbit by Proposition 3.2, its irreducibility follows from the irreducibility of GL(n)×GL(n). Since GXw is a fiber bundle over GGr×Fl(n) by Lemma 3.3 with fibers isomorphic to the codimension ℓ(w) Schubert cell Xw (as per Example 3.1), it has dimension dim(GGr×Fl(n))+dim(Fl(n))−ℓ(w), hence codimension ℓ(w).
2. (b)
The projection Cw→GGr×Fl(n) is a fiber bundle by Lemma 3.3. Lemma 2.24 shows that Cw(G(f),F∙) is a Schubert cell, irreducible of codimension ℓ(w), so the same is true of the fibers Cw(G(f),F∙). As in (a), this implies that codimCw=ℓ(w), and the irreducibility of GGr×Fl(n) and of Cw(G(f),F∙) implies that Cw is irreducible. The inclusion GXw⊆Cw is clear, so (a) forces equality.
3. (c)
By Lemma 3.3, both GXw→GGr and GXw∩(GGr×Fl(n)×Fl(n))→GGr are fiber bundles, and the proof of that lemma shows that the same maps provide local trivializations for both bundles simultaneously. This reduces (c) to the easy claim that if X and U⊆Y are spaces in which points are closed, and X×U is dense in X×Y, then {x}×Y={x}×U for any x∈X.
4. (d)
As in (c), this reduces by Lemma 3.3 to the claim that if X and Y are smooth manifolds and Y′⊆Y is a submanifold, then (X×Y′)∩({x}×Y)={x}×Y′ is transverse for x∈X. ∎
3.2. Cohomological formulas
In this subsection we show that [GXw] is represented by the back-stable double Schubert polynomial Sw(x;−y).
Lemma 3.6**.**
The class [GXw(G(f))] is represented by Sw(x;−y).
Proof.
Fix an invertible linear map f:Cn→Cn∗. Then GXw(G(f)) is the set of (E∙,F∙)∈Fl(n)×Fl(n) such that rk(FjfEi∗)=rk(Eif∗Fj∗)=rkw[i][j] for i,j∈[n]. Since codimGXw=ℓ(w) by Lemma 3.5(a), Fulton’s degeneracy locus formula (Theorem 2.22) applied to the sequence E1↪⋯↪Enf∗Fn∗↠⋯↠F1∗ gives [GXw(G(f))]=Sw(x′;y′), where xi′=c1((Ei/Ei−1)∗) and yi′=c1(Fi/Fi−1). Under the conventions from §2.1, xi′=xi and yi′=−yi. Lemma 3.5(c) shows that [GXw(G(f))]=[GXw(G(f))].
∎
For vector spaces V⊆W, let V⊥={α∈W∗:α∣V=0} be the annihilator of V.
Lemma 3.7**.**
If w∈Sn is vexillary, then GXw is the closure of the subset
[TABLE]
of Gr(n,2n)×Fl(n)×Fl(n).
Proof.
The image of the composition Fj↪CnfCn∗↠Ei∗ is the same as the image of G(f)∩(Fj⊕Cn∗)↪Cn⊕Cn∗↠Ei∗, where the last map sends Cn to [math] and restricts α∈Cn∗ to Ei.
Thus, (G(f),E∙,F∙)∈GXw if and only if
[TABLE]
for i,j∈[n]. Since dim(G(f)∩(Fj⊕Cn∗))=j because f is invertible, (4) is equivalent by the rank-nullity theorem to dim(G(f)∩(Fj⊕Ei⊥))=j−rkw[i][j]. If we enforce these rank conditions only for (i,j)∈Ess(D(w)), we get the set Bw∩(GGr×Fl(n)×Fl(n)), and Lemma 3.5 says that the closure of this set is GXw.
However, to show that Bw∩(GGr×Fl(n)×Fl(n))=Bw, we need to know that Bw has no components in the complement of GGr×Fl(n)×Fl(n). Since GGr is dense in Gr(n,2n), it suffices to show that Bw is irreducible. Given that w is vexillary, we can write \operatorname{Ess}(D(w))=\{(i_{1},j_{1})\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-3.05mm}\hbox{<}}\cdots\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-3.05mm}\hbox{<}}(i_{s},j_{s})\} by Lemma 2.15. We then have
[TABLE]
Lemma 3.3 implies that Bw is a fiber bundle over Fl(n)×Fl(n). For fixed (E∙,F∙)∈Fl(n)×Fl(n), the closure of the fiber
[TABLE]
is a Schubert variety in Gr(n,2n) with respect to the partial flag (5). Since Schubert varieties are irreducible, so too is each fiber Bw(E∙,F∙) and therefore Bw as well.
∎
Suppose w is vexillary and write \operatorname{Ess}(D(w))=\{(i_{1},j_{1})\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-3.05mm}\hbox{<}}\cdots\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-3.05mm}\hbox{<}}(i_{s},j_{s})\}. Lemma 3.7 shows that GXw is an example of a Grassmannian degeneracy locus as described in §2.4. Specifically, GXw is the closure of the locus of points in X=Gr(n,2n)×Fl(n)×Fl(n) over which dim(G∩Hμp)=kp for p∈[s], where:
▶
V is the trivial bundle Cn⊕Cn∗ over the first factor of X;
▶
G is the tautological bundle over the first factor of X;
▶
Hμp=Fjp⊕Eip⊥, where E∙ and F∙ are the tautological flags over the second and third factors of X, so μp=2n−rk(Fjp⊕Eip⊥)=n+ip−jp for p∈[s];
▶
kp=jp−rkw[ip][jp] for p∈[s], and k0=0;
▶
λk=ip−rkw[ip][jp] for k∈[ks], where p is such that kp−1<k≤kp.
Using this data and the setup of Theorem 2.25, we have
[TABLE]
where kp−1<k≤kp. Here, we have used the isomorphism E⊥≃(Cn/E)∗ to compute c(Eip⊥)=c((Cn/Eip)∗)=1/c(Eip∗)=(1+x1)⋯(1+xip)1. Under the identifications of §2.1, 1/c(G) is represented by ∑dhd(x−∖y−), so we will think of c(k)d as the power series hd(x−∞⋯ip∖y−∞⋯jp). Set c^{\prime}(k)=c(k)\bigr{\rvert}_{c(\mathcal{G})\to 1}, so c′(kp)d=hd(x1⋯ip∖y1⋯jp).
Lemma 3.8**.**
Fix a vexillary w and G(f)∈GGr. Then [GXw] is represented by det(c(k)λk+t−k)k,t∈[ℓ(λ)], and [GXw(G(f))] is represented by det(c′(k)λk+t−k)k,t∈[ℓ(λ)], where λ=sh(w).
Proof.
By Lemma 2.20, the partition λ associated to w above is the same as sh(w). This implies that GXw has the expected codimension ∣λ∣, given that ∣λ∣=∣sh(w)∣=∣D(w)∣=ℓ(w) and codimGXw=ℓ(w) by Lemma 3.5(a). As GXw is a Grassmannian degeneracy locus Ωλ with respect to the data described above, [GXw]=det(c(k)λk+t−k)k,t∈[ℓ(λ)] by Theorem 2.25.
Let i:Fl(n)×Fl(n)↪Gr(n,2n)×Fl(n)×Fl(n) be the inclusion (E∙,F∙)↦(G(f),E∙,F∙). The pullback i∗ on cohomology sends 1⊗b⊗c to b⊗c and a⊗b⊗c to [math] when a has positive degree, so i∗c(k)=c′(k). Now [GXw(G(f))]=[i−1(GXw)]=i∗[GXw]=det(c′(k)λk+t−k)k,t∈[ℓ(λ)], where the second equality holds by the transversality from Lemma 3.5(d).
∎
Theorem 3.9**.**
Let w be vexillary, with c(k), c′(k), and λ defined as above. As polynomials in x and y, Sw(x;−y) equals det(c′(k)λk+t−k)k,t∈[ℓ(λ)] and Sw(x;−y) equals det(c(k)λk+t−k)k,t∈[ℓ(λ)].
Proof.
The equality Sw(x;−y)=det(c′(k)λk+t−k)k,t∈[ℓ(λ)] is [10, Proposition 9.6(f)]; see also [30]. The permutation 1m×w is vexillary for any m∈N, with
[TABLE]
while the partition λ and sequence k∙ are independent of m. Lemma 3.8 therefore shows that [GX1m×w(G(f))] is represented by S1m×w(x;−y)=det(c′(k)λk+t−k)k,t∈[ℓ(λ)], where if kp−1<k≤kp then c′(k)=(1−y1)⋯(1−ym+jp)(1+x1)⋯(1+xm+ip). Now let c′′(k) be c′(k) with the alphabets x1⋯∞,y1⋯∞ replaced by x−m⋯∞,y−m⋯∞, so that
[TABLE]
This is the determinantal formula for Sw(x;−y) claimed in the theorem, since
[TABLE]
Lemma 3.8 and Theorem 3.9 imply that Sw(x;−y) represents [GXw] for vexillary w, but we want to show this for all w. Since w=w0=n(n−1)⋯21 is vexillary, the general result would follow if we knew that the classes [GXw] satisfied the same divided difference recurrence as Sw(x;−y) (Proposition 2.9). To that end, we recall a geometric interpretation of the divided difference operator ∂i; see [11, §10].
Let Fli(n) be the variety of partial flags
[TABLE]
where dimEp=p. Set X=Fl(n), and let X~ be the fiber product Fl(n)×Fli(n)Fl(n) with respect to the projection Fl(n)→Fli(n), so
[TABLE]
Let p1,p2:X~→X be the projections onto the two coordinates of X~. Under the Borel isomorphism (1), p2∗p1∗:H∗(Fl(n))→H∗(Fl(n)) corresponds to ∂i. We therefore also write ∂i for the operator p2∗p1∗. Suppose Y⊆Fl(n) is a closed subvariety. Then
[TABLE]
Now set X=Gr(n,2n)×Fl(n)×Fl(n) instead, let X~ be the fiber product X×Gr(n,2n)×Fli(n)×Fl(n)X, and let P1,P2:X~→X be the two canonical projections. Then P2∗P1∗ acts on H∗(Gr(n,2n)×Fl(n)×Fl(n)) as id⊗∂i⊗id.
Lemma 3.10**.**
For w∈Sn,
[TABLE]
Proof.
Lemma 3.3 implies that P1−1(GXw) and P2(P1−1(GXw)) are fiber bundles over GGr×Fl(n), with fibers p1−1(Xw) and p2(p1−1(Xw)). The map P2:P1−1(GXw)→P2(P1−1(GXw)) respects fibers, i.e. commutes with the two bundle projections to GGr×Fl(n). Moreover, after choosing an appropriate trivialization of these two bundles over an open set U⊆GGr×Fl(n) (as described in Lemma 3.3, for instance), this map P2 gets identified with id×p2:U×p1−1(Xw)→U×p2(p1−1(Xw)).
If ℓ(wsi)>ℓ(w), then dimp2(p1−1(Xw))≤dimXw, while if ℓ(wsi)<ℓ(w), then p2 is one-to-one from a dense subset of p1−1(Xw) onto a dense subset of Xwsi [11, §10.3, Lemma 8]. This plus the previous paragraph implies that if ℓ(wsi)>ℓ(w), then dimP2(P1−1(GXw))≤dimGXw, while if ℓ(wsi)<ℓ(w), then P2 is one-to-one from a dense subset of P1−1(GXw) onto a dense subset of GXwsi. Now apply (7).
∎
Theorem 3.11**.**
For w∈Sn, the back-stable double Schubert polynomial Sw(x;−y) represents the class [GXw].
Proof.
The theorem holds for w=w0 by Lemma 3.8 and Theorem 3.9, as w0 is vexillary. Therefore it holds for all w, since Sw(x;−y) and [GXw] satisfy the same recurrence with base case w=w0, by Proposition 2.9 and Lemma 3.10 respectively.
∎
4. Preliminaries on involutions
4.1. Involution Schubert polynomials
It will be convenient to write InO=In and InSp=In\textscfpf. Writing K for either O or Sp, the Bruhat-minimal element of InK is denoted 1nK, so 1nO=1n and 12rSp=(1,2)(3,4)⋯(2r−1,2r).
The Demazure product∘ is the unique associative product on Sn satisfying
[TABLE]
for any w∈Sn and simple transposition si=(i,i+1).
It is not hard to check that w−1∘z∘w is an involution whenever z is, and that for any z∈InK, there exists w∈Sn such that z=w−1∘1nK∘w.
Definition 4.2**.**
The set of atoms of z∈InK is
[TABLE]
A reduced involution word of z∈InK is a word a∈R(w) for some w∈AK(z). Equivalently, a is a minimal-length word a1⋯aℓ such that
[TABLE]
Let RK(z) be the set of reduced involution words of z∈InK, so RK(z)=⨆w∈AK(z)R(w). Let ℓK(z) be the length of any word in RK(z).
Example 4.3**.**
[TABLE]
and RO((1,3))={12,21} and AO((1,3))={231,312}. As a fixed-point-free example,
[TABLE]
and RSp((1,4)(2,3))={23,21} and ASp((1,4)(2,3))={3124,1342}. By contrast,
[TABLE]
and AO((1,4)(2,3))={4213,3412,2431}.
The next definition is identical to Definition 2.5 for Schubert polynomials save that reduced words have been replaced by reduced involution words. Let K be O or Sp.
Definition 4.4**.**
The back-stable involution Schubert polynomial of z∈InK is
[TABLE]
The involution Schubert polynomial of z is
[TABLE]
The involution Stanley symmetric function of z is
[TABLE]
Involution Schubert polynomials were introduced by Wyser and Yong [32] and further studied by Hamaker, Marberg, and the author in [12, 13, 14, 15], where involution Stanley symmetric functions were also investigated. They are homogeneous of degree ℓK(z). Like Schubert polynomials, they satisfy a divided difference recurrence, as do the back-stable versions. Recall that cyc(z) is the number of 2-cycles in z.
Proposition 4.5**.**
For z∈InK and i∈[n−1],
[TABLE]
Proof.
The fact that RK(1m×z)={(a1+m)⋯(aℓ+m):a1⋯aℓ∈RK(z)} for any m (even m, in the case K=Sp) implies SzK=limm→∞S1m×zK(x−m⋯n). Given that
[TABLE]
the lemma follows from Wyser and Yong’s result that the involution Schubert polynomials SzK satisfy the stated divided difference recurrence [32].
∎
For z∈InK, define {X}^{K}_{z}=\{E_{\bullet}\in\operatorname{Fl}(n):\text{\operatorname{rk}(E_{j}\xrightarrow{f}E_{i}^{*})=\operatorname{rk}z_{[i][j]}fori,j\in[n]}\} for some fixed invertible f:Cn→Cn∗, symmetric or skew-symmetric according to whether K is O(n) or Sp(n). The sets XzK for z∈InK are the K-orbits on Fl(n) [31].
[XyO]* is represented by 2cyc(y)SyO for y∈In, and [XzSp] is represented by SzSp for z∈In\textscfpf.*
4.2. Cohomology of LG(2n) and OG(2n)
For d≥0, let Qd be the symmetric function ∑a+b=dhaeb, so
[TABLE]
Also define P0=1 and Pd=21Qd for d>0; one checks that Pd has integer coefficients. Let ΓQ be the subring of Λ generated by the Qd, and ΓP the subring of Λ generated by the Pd, so ΓQ⊆ΓP.
Recall that LG(2n) is the closed subvariety of Gr(n,2n) of subspaces isotropic with respect to the skew-symmetric form ((v1,ω1),(v2,ω2))−=ω1(v2)−ω2(v1). The subvariety of Gr(n,2n) consisting of subspaces isotropic with respect to the symmetric form ((v1,ω1),(v2,ω2))+=ω1(v2)+ω2(v1) has two components, and OG(2n) is defined to be the component containing Cn.
If E⊆Cn is a subspace, the subspace E⊕E⊥⊆Cn⊕Cn∗ is isotropic under (−,−)− and (−,−)+. As in §2.1, the maps Fl(n)→LG(2n) and Fl(n)→OG(2n) defined by E∙↦Ei⊕Ei⊥ induce maps on cohomology with
[TABLE]
where G is the tautological bundle on LG(2n) or OG(2n). This suggests identifying cd(G∗) with Qd∈ΓQ, and indeed Pragacz [26] showed that sending Qd↦cd(G∗) induces a ring isomorphism \Gamma_{Q}/(\text{Q_{d}ford>n})\to H^{*}(\operatorname{LG}(2n)).
The analogous map for H∗(OG(2n)) is well-defined and injective but not surjective. However, it extends to an isomorphism \Gamma_{P}/(\text{P_{d}ford>n})\to H^{*}(\operatorname{OG}(2n)), again sending 2Pd=Qd↦cd(G∗) for d>0 [26].
4.3. Schur P- and Q-functions
Let λ be a finite sequence of nonnegative integers of length ℓ=ℓ(λ). Define
The Schur Q-function of λ is the symmetric function defined by:
(i)
Q∅=1.
2. (ii)
Q(λ1)=Qλ1=∑a+b=λ1haeb as defined in §4.2.
3. (iii)
Q(λ1,λ2)=Qλ1Qλ2+2∑p=1λ2(−1)pQλ1+pQλ2−p if (λ1,λ2)=(0,0), and Q(0,0)=0.
4. (iv)
Qλ=pf(Q(λi+,λj+))i,j∈[ℓ(λ+)],
where pf(A) is the Pfaffian of a matrix A.
The Schur P-function of λ is Pλ=def2−ℓ(λ)Qλ.
A priori Pλ is only a symmetric function with rational coefficients, but in fact it has integral coefficients. Note that we have not required that λ be a partition or that all its parts be positive, and part (iv) above may involve Q-functions indexed by such λ. However, one can show that
[TABLE]
which implies that the matrix in (iv) is skew-symmetric of even size. One might think of (iv) as analogous to the Jacobi-Trudi formula expressing a Schur function as a determinant of single-row Schur functions.
Example 4.8**.**
[TABLE]
Definition 4.9**.**
A partition λ is strict if λ1>⋯>λℓ>0.
Clearly Qλ∈ΓQ and Pλ∈ΓP, and in fact \{Q_{\lambda}:\text{\lambda strict}\} and \{P_{\lambda}:\text{\lambda strict}\} are Z-bases for ΓQ and ΓP. Pragacz showed that under the isomorphisms described in §4.2, the Qλ for λ strict with λ1≤n represent the classes of the Schubert varieties in LG(2n), and the Pλ for λ strict with λ1<n represent the classes of the Schubert varieties in OG(2n) [26].
Let λ be a strict partition. The shifted Young diagram of λ is
[TABLE]
We draw Dλ′ as a set of boxes in matrix coordinates, with (1,1) at the upper left:
[TABLE]
Definition 4.10**.**
A filling T of the shifted diagram of λ by entries from the alphabet {1′<1<2′<2<⋯} is a marked shifted standard tableau (of shape λ) if
•
the entries are weakly increasing reading down columns and across rows;
•
no column contains the same unprimed letter twice;
•
no row contains the same primed letter twice.
More generally, if S={⋯<a−1<a0<a1<⋯} is any set of integers, we let ShSYT′(λ,S) denote the set of marked shifted standard tableaux of shape λ on the alphabet {⋯a−1′<a−1<a0′<a0<a1′<a1<⋯}.
If i is an unprimed integer we set ⌈i⌉=⌈i′⌉=i and ε(i)=1 and ε(i′)=−1. If T is a filling of Dλ′ and (i,j)∈Dλ′, let T(i,j) be the entry of T in row i and column j.
Example 4.11**.**
\young(1′34′45,:4′46′,::56′) is a marked shifted standard tableau of shape (5,3,2). We have T(2,4)=6′, so ⌈T(2,4)⌉=6 and ε(T(2,4))=−1.
The monomial expansion of a Schur Q-function can be expressed in terms of shifted tableaux [25, §III.8]:
[TABLE]
We will need certain multivariate generalizations of Schur Q-functions introduced by Ivanov; the next definition is [18, Theorem 4.3].
Definition 4.12**.**
Let λ be a strict partition and t a sequence of at least λ1 indeterminates. The multiparameter Schur Q-function associated to λ is
[TABLE]
4.4. Degeneracy locus formulas
The Kempf-Laksov formula (Theorem 2.25) expressed the class of a generic Grassmannian degeneracy locus as a determinant in certain Chern classes, which reduces to the Jacobi-Trudi formula in a special case. We will need an analogous formula for isotropic Grassmannian degeneracy loci Ω. These formulas, originally due to Kazarian [19] and expanded upon by Anderson and Fulton [3], express [Ω] as a Pfaffian in appropriate Chern classes. These Pfaffians reduce to the Pfaffian formulas for Qλ and Pλ in special cases. In parallel with Definition 4.7, suppose c(1),…,c(ℓ) are formal power series with constant term 1 and that λ is a finite sequence of nonnegative integers of length ℓ, and define Qλ(c(1),…,c(ℓ)) by
(i)
Q∅()=1.
2. (ii)
Q(λ1)(c(1))=c(1)λ1
3. (iii)
Q(λ1,λ2)(c(1),c(2))=c(1)λ1c(2)λ2+2∑p=1λ2(−1)pc(1)λ1+pc(2)λ2−p if (λ1,λ2)=(0,0), while Q(0,0)(c(1),c(2))=0.
4. (iv)
When ℓ(λ+)>ℓ(λ)=ℓ, the formula in (iv) refers to c(ℓ+1), which we take to be 1. With this convention, Q(λi,0)(c(i),c(ℓ+1))=Q(λi)(c(i)). Of course, the c(i) must satisfy certain relations in order for the matrix in (iv) to be skew-symmetric, but these relations will always hold for us.
Suppose V is a rank 2n vector bundle over a smooth variety X with a rank n subbundle G and a flag of subbundles Hμ1⊆⋯⊆Hμs⊆V, where rkHi=rkn−i+1. Also assume that V is equipped with a nondegenerate skew-symmetric form, and that G and Hi are isotropic with respect to this form. Fix a sequence k∙=(k1<⋯<ks) of integers, setting k0=0. For each k∈[ks], define λk=μp+kp−k and c(k)=c(V)/(c(G)c(Hμp)), where p is such that kp−1<k≤kp. The corresponding Lagrangian Grassmannian degeneracy locusΩLG is the closure of the locus in X over which dim(G∩Hμp)=kp for each p∈[s].
If λ is a strict partition and ΩLG has codimension ∣λ∣, then [ΩLG]=Qλ(c(1),…,c(ks)).
These Pfaffians can be expressed in terms of the multiparameter Schur Q-functions. Suppose the partial flag of isotropic bundles Hμ1⊆⋯⊆Hμs⊆V extends to a complete isotropic flag 0=Hn+1⊆Hn⊆⋯⊆H1 in V. Set t1′=0 and ti+1′=c1(Hi/Hi+1) for i∈[n] and ti′=0 for i>n+1. The multiparameter Schur Q-function Qλ(x;t) is in ΓQ[t1,t2,…] [18, Proposition 2.12]. The relations between the generators Q1,Q2,… of ΓQ are described in [2], and these relations also hold between the Chern classes (c(G)c(H1)c(V))d where d≥1, so there is a well-defined ring map ΓQ[t1,t2,…]→H∗(X) sending Qd↦(c(G)c(H1)c(V))d and ti↦ti′.
After identifying Qd(x1,x2,…) with (c(G)c(H1)c(V))d as just described, Qλ(c(1),…,c(ks))=Qλ(x;−t′).
We will also need an orthogonal version of Theorem 4.13, which will be a little more complicated. As above, let V be a rank 2n vector bundle over a smooth variety X with a rank n subbundle G and a flag of subbundles Hμ1⊆⋯⊆Hμs⊆V—but now we use the slightly different convention rkHi=n−i. Assume that V is equipped with a nondegenerate symmetric form, and that G and Hi are isotropic with respect to this form. Fix a sequence k∙=(k1<⋯<ks) of integers, setting k0=0. Just as above, for each k∈[ks], define λk=μp+kp−k where p is such that kp−1<k≤kp. The corresponding orthogonal Grassmannian degeneracy locusΩOG is the closure of the locus in X over which dim(G∩Hμi)=kp for each p∈[s].
If c(1),…,c(ℓ) are formal power series with constant term 1 over a ring in which 2 is invertible, define Pλ(c(1),…,c(ℓ))=2−ℓ(λ)Qλ(c(1),…,c(ℓ)). For simplicity in stating the next theorem, we assume that the flag H∙ includes a maximal isotropic bundle H0. We also assume that X is connected, which implies that δ=defdim(Gx∩(H0)x)(mod2) is constant on X. For k∈[ks], let p be such that kp−1<k≤kp, and define
If λ is a strict partition and ΩOG has the expected codimension ∣λ∣, then [ΩOG]=Pλ(c(1),…,c(ks)).
Remark 4.16**.**
Except in degree μp=λkp, the classes c(k) agree with c(V)/(c(G)c(Hμp)), which was the definition of c(k) used in Theorem 4.13. The correction terms (−1)δcμp(H0/Hμp) are independent of the choice of H0, which we will exploit later by choosing different H0 for different k.
4.5. Vexillary involutions
In this subsection we give definitions and prove lemmas analogous to those of §2.3 for vexillary involutions.
Definition 4.17**.**
We define two analogues of the Rothe diagram for y∈In:
[TABLE]
One can show that the size of DK(y) is ℓK(y) and that the K-orbit XyK is defined by rank conditions coming from Ess(DK(y)) (cf. Lemma 2.24) [13].
The next lemma follows from Lemma 2.15 and the fact that D(w−1)={(j,i):(i,j)∈D(w)}.
Lemma 4.18**.**
*An involution y is vexillary if and only if Ess(DO(y)) is a chain in the partial order
↗
≤
.*
Definition 4.19**.**
Let y∈In. We define two involution codes cO(y) and cSp(y) as the row lengths of DO(y) and cSp(y) respectively. That is,
[TABLE]
We also define two involution shapes shO(y) and shSp(y), whose conjugates are respectively obtained by sorting the nonzero entries of cO(y) and of cSp(y).
Lemma 4.20**.**
Suppose y∈In is vexillary, and let j′ be maximal such that (i′,j′)∈Ess(DO(y)) for some i′. Then
[TABLE]
and
[TABLE]
Proof.
We proceed by proving a series of claims:
(a)
If i≤j′, then y(i)=i or y(i)>j′: If i=j′ this holds since (i′,j′)∈D(y) implies y(j′)>i′≥j′, so assume i<j′. Note that y(i)=j′ because otherwise (i′,j′)∈/D(y). But now if i<y(i)<j′, then we have i<y(i)<j′<y(j′), which gives a 2143 pattern in y, contradicting the assumption that y is vexillary. If y(i)<i, the same contradiction occurs in positions y(i)<i<j′<y(j′), so we must have y(i)=i or else y(i)>j′.
2. (b)
If i≤j′, then {j∈[i]:(i,j)∈DO(y)}={j∈[i]:y(j)=j}: If y(j)=j, then (i,j)∈/DO(y) for any i. Conversely, suppose y(j)=j for some j∈[i]. Since j≤i≤j′, (a) implies that y(j)>j′ and y(i)≥min(i,j′)=i. But then y(j)>i and y(i)≥j, so (i,j)∈DO(y).
3. (c)
If i>j′, then ciO(y)=ci(y): If y(i)>i, then (i,i)∈DO(y). But then the connected component of (i,i) in DO(y) contains some essential set element (a,b) with b≥i>j′, contradicting the choice of j′. It follows that y(i)≤i, which implies that ciO(y)=ciSp(y)=ci(y).
Taking cardinalities, part (b) shows ciO(y)=∣{j∈[i]:y(j)=j}∣. It also shows that {j∈[i−1]:(i,j)∈DSp(y)}={j∈[i−1]:y(j)=j}, so ciSp(y) is the size of ∣{j∈[i−1]:y(j)=j}∣ as claimed.
∎
Lemma 4.21**.**
Suppose y∈In is vexillary. Write \operatorname{Ess}(D^{\operatorname{O}}(y))=\{(i_{1},j_{1})\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-3.05mm}\hbox{<}}\cdots\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-3.05mm}\hbox{<}}(i_{s},j_{s})\}, and define kp=jp−rky[ip][jp] for p∈[s]. Then {c1O(y),…,cnO(y)}∖{0}=[ks], and if k∈[ks]∖{k1,…,ks} there is a unique i∈[n] with ciO(y)=k.
Proof.
Recall the notation DiS=DiS(y)={j∈S:(i,j)∈D(y)}.
(a)
Djs[js]=Dis[js]: The northwest corner closure property of D(y) mentioned in the proof of Lemma 2.20 implies that (js,js)∈D(y), given that D(y) contains (is,js) and (js,is). If (is,j)∈D(y) and j≤js, then (i_{s},j)\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-2.65mm}\hbox{\leq}}(j_{s},j_{s}), so the same closure property implies (js,j)∈D(y); thus, Dis[js]⊆Djs[js].
Conversely, suppose (js,j)∈D(y) but (is,j)∈/D(y) for some j≤js. Then we must have js<y(j)≤is, so Dy(j)[j,∞)=∅. But this means that the portion of the connected component of (js,js)∈DO(y) southeast of (js,js) lies entirely above row y(j); in particular, there is an essential set cell in a row strictly above row y(j). Given that y(j)≤is, this contradicts the maximality of (is,js)∈Ess(DO(y)) with respect to
↗
≤
. Thus, Djs[js]⊆Dis[js] as well.
2. (b)
If k∈[ks] then k=ciO(y) for some i: Part (a) shows that cjsO(y)=∣Djs[js]∣=∣Dis[js]∣=ks. By Lemma 4.20, the sequence c1O(y),c2O(y),…,cjsO(y) is weakly increasing with consecutive differences in {0,1}, and starts with c1O(y)∈{0,1} and ends with cjsO(y)=ks. Thus, {ciO(y):i∈[n]}∖{0}⊇[ks].
3. (c)
If i∈[n] then ciO(y)∈{0}∪[ks]: We saw in part (b) that if i≤js then ciO(y)≤cjsO(y)=ks. Suppose i>js. By the maximality of (is,js)∈Ess(DO(y)), there cannot be any cells in DO(y) right of column js, so ci(y)=∣Di[js]∣. If (i,j)∈D(y) with j≤js, then (i,j)\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-2.65mm}\hbox{\leq}}(j_{s},j_{s}), so the northwest closure property of D(y) implies (js,j)∈D(y) also. Thus ciO(y)=∣Di[js]∣≤∣Djs[js]∣=cjsO(y)=ks.
4. (d)
If k∈[ks]∖{k1,…,ks} then k=ciO(y) for a unique i: Suppose that ciO(y)=ci′O(y)=k where i<i′. Since k∈/{k1,…,ks}, we cannot have k∈{c1(y),…,cn(y)} by Lemma 2.19, and so Lemma 4.20 forces i<i′≤js. Lemma 4.20 also says that since ciO(y)=ci′O(y), every member of [i+1,i′] is a fixed point of y. But if y(i′)=i′, then k=ci′O(y)=ci′(y), so k∈{k1,…,ks} by Lemma 2.19, a contradiction. ∎
Lemma 4.22**.**
Suppose y∈In is vexillary. Write \operatorname{Ess}(D^{\operatorname{O}}(y))=\{(i_{1},j_{1})\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-3.05mm}\hbox{<}}\cdots\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-3.05mm}\hbox{<}}(i_{s},j_{s})\}, and define kp=jp−rky[ip][jp] for p∈[s] and k0=0. Then ℓ(shO(y))=ks, and shO(y)k=ip−jp+1+kp−k for k∈[ks] where kp−1<k≤kp.
Proof.
Lemma 4.21 implies that ℓ(shO(y))=ks. Set Rp={i∈[jp,ip]:(i,jp)∈/DO(y)} for each p. We first show that
[TABLE]
Note that the set on the lefthand side has size shO(y)kp, while the set on the righthand side has size ip−jp+1.
(a)
If i∈[jp−∣Rp∣,jp−1], then y(i)=i: Assume for the sake of contradiction that i∈[jp−∣Rp∣,jp−1] has y(i)=i. Note that if i′∈Rp, then y(i′)<jp≤i′, so y(i′) is not a fixed point of y. This means {y(i′):i′∈Rp}=[jp−∣Rp∣,jp−1], since the set on the right contains a fixed point by assumption. So, take i′∈Rp such that y(i′)∈/[jp−∣Rp∣,jp−1]. Then y(i′)<jp−∣Rp∣≤i. But if this happens, then i<jp≤i′<ip<y(jp) and y(i′)<jp−∣Rp∣≤y(i)=i<jp<y(ip), so y contains a 2143 pattern in positions i,i′,ip,y(jp), a contradiction.
2. (b)
If i≤jp, then ciO(y)≥kp if and only if i∈[jp−∣Rp∣,jp−1].
Part (a) of the proof of Lemma 4.20 shows that rky[jp][jp]=∣{j≤jp:y(j)=j}∣, so that cjpO(y)=jp−rk[jp][jp](y) by the same lemma. Also, ∣Rp∣=rky(jp,ip][jp], so
[TABLE]
Lemma 4.20 and part (a) now show that cjp−∣Rp∣+rO(y)=kp+r for 0≤r≤∣Rp∣ and that cjO(y)<cjp−∣Rp∣O(y)=kp for j<jp−∣Rp∣.
3. (c)
If i>js≥jp, then ciO(y)≥kp if and only if i∈[jp,ip]∖Rp: Lemma 4.20 says ciO(y)=ci(y), so ciO(y)≥kp if and only if (i,jp)∈D(y) and i≤ip by Lemma 2.19.
4. (d)
If js≥i>jp, then ciO(y)≥kp if and only if i∈[jp,ip]∖Rp: On the one hand, for any such i we have ciO(y)≥cjpO(y)=kp+∣Rp∣≥kp, using (11) and Lemma 4.20. On the other hand, we claim that all such i are in [jp,ip]∖Rp. Suppose otherwise, so that i∈Rp, and hence y(i)<jp. Then Di[jp,∞)=∅, so the portion of the connected component of (jp,jp)∈DO(y) southeast of (jp,jp) lies entirely above row i; in particular, there is an essential set cell in a row strictly above row i. Given that i≤js≤is, this contradicts the maximality of (is,js)∈Ess(DO(y)) with respect to
↗
≤
.
We have proven equation (10), which implies shO(y)kp=ip−jp+1 by taking cardinalities. If kp−1<k<kp, then the fact that k appears exactly once in cO(y) (Lemma 4.21) implies that shkO(y)=shk+1O(y)+1, so that shkO(y)=shkpO(y)+kp−k.
∎
We conclude with an analogue of the last lemma for DSp(y) instead of DO(y).
Definition 4.23**.**
Say y∈In is Sp-vexillary if it is vexillary and Ess(DSp(y)) is a chain under
↗
≤
.
The involution y=(1,3)(2,5) is vexillary but has
[TABLE]
with essential sets highlighted in black, so y is not Sp-vexillary. The involution y=(1,2)(3,4) is not vexillary, but Ess(DSp(y))=∅ is a chain.
Recall that 12rSp=(1,2)(3,4)⋯(2r−1,2r) is the minimal-length element of I2r\textscfpf.
Lemma 4.24**.**
Suppose y′ is Sp-vexillary and that y=y′×12rSp or y=12rSp×y′. Write \operatorname{Ess}(D^{\operatorname{Sp}}(y))=\{(i_{1},j_{1})\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-3.05mm}\hbox{<}}\cdots\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-3.05mm}\hbox{<}}(i_{s},j_{s})\}, and define kp=jp−rky[ip][jp] for p∈[s] and k0=0. Then ℓ(shSp(y))=ks, and shSp(y)k=ip−jp+kp−k for k∈[ks] where kp−1<k≤kp.
Proof.
Replacing y′ with y′×12rSp does not change essential sets or the sequences ip, jp, and kp. Replacing y′ with 12rSp×y′ replaces ip by ip+2r and jp by jp+2r, and does not change kp. In both cases, the partition shSp(y′) does not change, nor do the quantities ip−jp+kp−k. The truth of the lemma for y′ would therefore imply it for y, so we can assume that y itself is Sp-vexillary.
Let Ess(DO(y))={(i1′,j1′),…,(it′,jt′)} and kp′=jp′−rky[ip′][jp′] for p∈[t]. By Lemma 4.20, c1Sp(y)=0 and ciSp(y)=ci−1O(y) if 1<i≤jt′, while ciSp(y)=ciO(y) if i>jt′. Thus, shSp(y)t is obtained from shO(y)t by deleting a part of size cjt′O(y). Equivalently, shSp(y) is obtained from shO(y) by decrementing its first cjt′O(y) parts, deleting any [math]’s that result. But part (a) of the proof of Lemma 4.21 says cjt′O(y)=kt′=ℓ(shO(y)), so shSp(y) is obtained from shO(y) by decrementing every part.
Given this, Lemma 4.22 shows that for k∈[kt′] we have shSp(y)k=ip′−jp′+kp′−k where kp−1′<k≤kp′, ignoring [math]’s. To complete the proof, we must show that ks<k≤kt′ if and only if shO(y)k=1, so that shSp(y) has length ks as claimed, and that if p≤s then ip′−jp′+kp′=ip−jp+kp. We must also see that kp−1′<k≤kp′ if and only if kp−1<k≤kp whenever k≤ks; this will follow from the fact that s≤t and ks≤ks′, which will be clear in each case.
(a)
Suppose Ess(DO(y)) contains no cell on the main diagonal. Then Ess(DO(y))=Ess(DSp(y)), so s=t and kp=kp′ and (ip,jp)=(ip′,jp′) for p∈[s]. Also, shO(y)k=ip′−jp′+kp′−k+1>1 for all k∈[ks] since (ip′,jp′) is always strictly below the main diagonal.
2. (b)
Suppose Ess(DO(y)) contains a cell on the main diagonal. Since Ess(DO(y)) is a chain under
↗
≤
, it contains at most one diagonal cell, and this cell (if it exists) must be (is′,js′)=(js′,js′).
(i)
Suppose (js′,js′−1)∈DO(y). Then Ess(DSp(y))=Ess(DO(y))∖{(js′,js′)}∪{(js′,js′−1)}. Thus, s=t and kp=kp′ and (ip,jp)=(ip′,jp′) for p<s. Moreover, (is,js)=(js′,js′−1), and (js′,js′)∈D(y) implies rky[js′][js′]=rky[js′][js′−1], so ks=ks′−1 and is′−js′+ks′=is−js+ks. Since shO(y) is a strict partition, shO(y)ks′=is′−js′+1=1 is its only part of size 1, so it does hold that shO(y)k=1 if and only if k∈(ks,ks′]=(ks′−1,ks′].
2. (ii)
Suppose (js′,js′−1)∈/DO(y). Then Ess(DSp(y))=Ess(DO(y))∖{(js′,js′)}, so t=s+1. Lemma 2.21 shows that ks=kt′−1, and just as in part (ii) we conclude that shO(y)k=1 if and only if k∈(ks,kt′]. ∎
5. GL(n)-orbits on LG(2n)×Fl(n) and OG(2n)×Fl(n)
Let g∈GL(n) act on U∈Gr(n,2n) by g⋅U=(g⊕(g∗)−1)(U). This defines a GL(n)-action on LG(2n) and on OG(2n).
5.1. Description of orbits
Define
[TABLE]
The next lemma shows that GLG=LG(2n)∩GGr and GOG=OG(2n)∩GGr, so GLG and GOG are open sets in LG(2n) and OG(2n).
Lemma 5.1**.**
G(f)∈LG(2n)* if and only if f:Cn→Cn∗ is symmetric, and G(f)∈OG(2n) if and only if f:Cn→Cn∗ is skew-symmetric.*
Proof.
It is easy to check that (G(f),G(f))−=0 if and only if f is symmetric, and (G(f),G(f))+=0 if and only if f is skew-symmetric, so we just need to see in the latter case that G(f) is actually in OG(2n). Let
[TABLE]
so OG(2n) is the irreducible component of Z containing Cn⊕0. Two isotropic subspaces U1,U2∈Z are in the same component of Z if and only if dim(U1∩U2)≡n(mod2). The fact that f is invertible and skew-symmetric implies both that n is even and that Cn∩G(f)=0, so indeed Cn and G(f) are in the same component of Z.
∎
Given y∈In, let LGXy⊆GLG×Fl(n) be
[TABLE]
Given z∈In\textscfpf, let OGXz⊆GOG×Fl(n) be
[TABLE]
More generally, for any y∈In, it will be convenient to define OGXy⊆OG(2n)×Fl(n) as the closure of
[TABLE]
The proof of Lemma 5.4(b) below implies that if y is fixed-point-free, the two definitions of OGXy agree. If y is not fixed-point-free, the notation is slightly misleading, since we are not taking OGXy to be the closure of OGXy, and indeed we leave the latter undefined. Replacing Ess(DSp(y)) with [n]×[n] in the general definition of OGXy above gives the empty set if y is not fixed-point-free.
Proposition 5.2**.**
The GL(n)-orbits on GLG×Fl(n) are the sets LGXy for y∈In. The GL(n)-orbits on GOG×Fl(n) are the sets OGXz for z∈In\textscfpf.
Proof.
This follows by an argument analogous to the proof of Proposition 3.2. Sending G(f) to the matrix of f defines an isomorphism from GLG to the space of invertible symmetric matrices GLsym(n), under which the GL(n)-action on GLG corresponds to the action of g∈GL(n) on A∈GLsym(n) by g⋅A=(gt)−1Agt. It then suffices to show that the Bn+-orbits on GLsym(n) are the sets
[TABLE]
for y∈In. Similarly, we must see that the Bn+-orbits on the space of invertible skew-symmetric matrices GLssym(n) are the sets
[TABLE]
for z∈In\textscfpf. These statements were proven in [4, 9] and in [28].
∎
In the OGXz case, Proposition 5.2 is not very interesting when n is odd, given that both In\textscfpf and GOG are empty. One might instead take GOG′ to be the set of graphs of skew-symmetric maps Cn→Cn∗ of maximal rank 2⌊n/2⌋, in which case the GL(n)-orbits on GOG′×Fl(n) are indexed by maximal rank skew-symmetric n×n(0,1,−1)-matrices with at most one ±1 in each row and column.
Lemma 5.3**.**
Each LGXy and OGXy is a fiber bundle over GLG and GOG, respectively, as well as over Fl(n).
for fixed f, either symmetric or skew-symmetric as appropriate, were called XyO and XzSp in §4.1. Since GL(n) acts transitively on GLG with stabilizers isomorphic to O(n), and transitively on GOG with stabilizers isomorphic to Sp(n), Proposition 5.2 recovers the fact mentioned in §4.1 that these fibers are the O(n)- and Sp(n)-orbits on Fl(n).
Lemma 5.4**.**
Let y∈In and z∈In\textscfpf.
(a)
LGXy* and OGXz are irreducible, with codimensions ℓO(y) and ℓSp(z) respectively.*
2. (b)
LGXy* is the closure in LG(2n)×Fl(n) of*
[TABLE]
and OGXz is the closure in OG(2n)×Fl(n) of
[TABLE]
3. (c)
LGXy(G(f))=LGXy(G(f))* for any G(f)∈GLG. Also, if E∙∈Fl(n) then LGXy(E∙)=LGXy(E∙). Likewise for OGXz.*
4. (d)
The intersection LGXy(G(f))=LGXy∩({G(f)}×Fl(n)×Fl(n)) is transverse, and likewise for OGXz.
Proof.
(a)
Given that LGXy and OGXz are orbits of an action of the irreducible group GL(n) by Proposition 5.2, they are irreducible. Just as in Lemma 3.5(a), their codimensions are the codimensions of the fibers LGXy(G(f))=XyO and OGXz(G(f))=XzSp over Fl(n), which are ℓO(y) and ℓSp(z) respectively. This last statement is implicit in Theorem 4.6, for instance, since codimXyO=deg[XyO]=degSyO=ℓO(y) and likewise for XzSp.
2. (b)
Proceed as in the proof of Lemma 3.5(b), replacing Lemma 2.24 with the following argument. Let SMy=,ess, SMy≤,ess, SMy=, and SMy≤ be the sets of symmetric g∈Hom(Cn,Cn∗) for which the ranks rk(Ci↪CngCn∗↠Cj∗) are either equal to or at most the ranks rky[i][j], either for (i,j)∈Ess(DO(y)) or for all (i,j)∈[n]×[n]. We must see that SMy==SMy=,ess. The inequalities rk(EjfEi∗)≤rky[i][j] for (i,j)∈Ess(DO(y)) logically imply those for all (i,j)∈[n]×[n] [13, Proposition 3.16]. Thus, SMy≤,ess=SMy≤, while SMy==SMy≤ by [4, Lemma 5.2]. Since SMy=⊆SMy=,ess⊆SMy≤,ess by definition, we are done. The same argument works in the skew-symmetric case, using [9].
3. (c)
In this subsection we show that [LGXy] and [OGXz] are represented by the back-stable involution Schubert polynomials 2cyc(y)SyO and SzSp. The proof follows the same outline as that of Theorem 3.11, so we mostly refer back to the results of §3.2, indicating differences when necessary.
Lemma 5.5**.**
If y∈In is vexillary then LGXy is the closure of the locus
[TABLE]
If y∈In is Sp-vexillary, then OGXy is the closure of
[TABLE]
Proof.
Apply the argument of Lemma 3.7, using Lemma 5.4(b) and replacing Schubert varieties in Gr(n,2n) with Schubert varieties in LG(2n) or OG(2n).
∎
Suppose y∈In is vexillary, so we can write \operatorname{Ess}(D^{\operatorname{O}}(y))=\{(i_{1},j_{1})\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-3.05mm}\hbox{<}}\cdots\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-3.05mm}\hbox{<}}(i_{s},j_{s})\}. Lemma 5.5 shows that LGXy is an example of a Lagrangian Grassmannian degeneracy locus as described in §4.4. To be specific:
▶
V is the trivial bundle Cn⊕Cn∗ over the first factor of X=LG(2n)×Fl(n);
▶
G is the tautological bundle over the first factor of X;
▶
Hμp=Ejp⊕Eip⊥, where E∙ is the tautological flag of bundles over the second factor of X, so μp=n−rk(Ejp⊕Eip⊥)+1=ip−jp+1 for p∈[s].
▶
kp=jp−rky[ip][jp] for p∈[s].
▶
λk=μp+kp−k for k∈[ks], where p is such that kp−1<k≤kp.
Using this data,
[TABLE]
where kp−1<k≤kp. Identify 1/c(G) with ∑dQd(x−) as in §4.2, so
[TABLE]
Set c^{\prime}(k)=c(k)\bigr{\rvert}_{c(\mathcal{G})\to 1}, so c′(k)d=hd(x1⋯ip∖x1⋯jp).
Lemma 5.6**.**
Let y∈In be vexillary, and fix G(f)∈GLG. The class [LGXy] is represented by Qλ(c(1),…,c(ks)), and [LGXy(G(f))] is represented by Qλ(c′(1),…,c′(ks)), where λ=shO(y).
Proof.
Lemma 4.22 shows that λ as defined above equals shO(y), and that this partition is strict. By Lemma 5.4(a), codimLGXy=ℓ(y)=∣shO(y)∣=∣λ∣. This is the expected codimension for LGXy as a Lagrangian Grassmannian degeneracy locus ΩLG with respect to the data described above, so [LGXy]=Qλ(c(1),…,c(ks)) by Theorem 4.13. As in the proof of Lemma 3.8, this implies that [LGXy(G(f))]=Qλ(c′(1),…,c′(ks)).
∎
Theorem 5.7**.**
Let y∈In be vexillary. As polynomials in x, Qλ(c′(1),…,c′(ks)) equals 2cyc(y)SyO(x) and Qλ(c(1),…,c(ks)) equals 2cyc(y)SyO(x), where λ=shO(y).
Proof.
Fix G(f)∈LG(2n). If y is vexillary, so is y×1, and DO(y×1)=DO(y). Thus, the single polynomial Qλ(c′(1),…,c′(ks)) represents the classes [OGXy×1m(G(f))] for all m. By Theorem 4.6, 2cyc(y)SyO represents [OGXy×1m(G(f))] when m=0, and in fact for all m since SyO=Sy×1O ([32, Theorem 2]). By Lemma 2.11, we conclude that Qλ(c′(1),…,c′(ks))=2cyc(y)SyO. The same limiting argument given in the proof of Theorem 3.9 now yields Qλ(c(1),…,c(ks))=2cyc(y)SyO.
∎
In [12], Pfaffian formulas for SyO were given in the case that y is I-Grassmannian, meaning that Ess(DO(y))⊆{m}×N for some m (see §6). Theorem 5.7 generalizes those formulas to all vexillary y, but is an improvement even when y is I-Grassmannian: [12] expresses SyO as a Pfaffian of polynomials Sy′O where shO(y′) has two rows, but does not give explicit formulas for the two-row case.
Lemma 5.8**.**
For y∈In and z∈In\textscfpf,
[TABLE]
and
[TABLE]
Proof.
Fixing an invertible and symmetric or skew-symmetric f as appropriate, Wyser and Yong proved these recurrences hold for the classes [LGXy(G(f))] and [OGXz(G(f))] (Proposition 4.5 and Theorem 4.6). The lemma then follows by the argument of Lemma 3.10.
∎
Theorem 5.9**.**
For y∈In, the back-stable involution Schubert polynomial 2cyc(y)SyO represents the class [LGXy].
Proof.
Theorem 5.7 implies that 2cyc(w0)Sw0O represents [LGXw0], so the theorem follows from the matching divided difference recurrences of Proposition 4.5 and Lemma 5.8.
∎
Suppose y∈In is Sp-vexillary, so we can write \operatorname{Ess}(D^{\operatorname{Sp}}(y))=\{(i_{1},j_{1})\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-3.05mm}\hbox{<}}\cdots\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-3.05mm}\hbox{<}}(i_{s},j_{s})\}. Lemma 5.5 implies that OGXy is an example of an orthogonal Grassmannian degeneracy locus as described in §4.4. To be specific:
▶
V is the trivial bundle Cn⊕Cn∗ over the first factor of X=OG(2n)×Fl(n);
▶
G is the tautological bundle over the first factor of X;
▶
Hμp=Ejp⊕Eip⊥, where E∙ is the tautological flag of bundles over the second factor of X, so μp=n−rk(Ejp⊕Eip⊥)=ip−jp for p∈[s].
▶
kp=jp−rky[ip][jp] for p∈[s].
▶
λk=μp+kp−k for k∈[ks], where p is such that kp−1<k≤kp.
For each p, H0=Ejp⊕Ejp⊥ is a maximal isotropic bundle containing Ejp⊕Eip⊥. In accordance with §4.4, we define
[TABLE]
for k∈[ks], where kp−1<k≤kp. Here, the quantity δ=dim(Gx∩(Ejp⊕Ejp⊥)x)(mod2) is independent of x∈X since X is connected, and we compute it by choosing x=(Cn⊕0,E∙) for some E∙∈Fl(n), so δ=dim((Cn⊕0)∩(Ejp⊕Ejp⊥))=jp.
The choice of H0 as Ejp⊕Ejp⊥ was somewhat arbitrary, and could be replaced by Ej⊕Ej⊥ for any j∈[jp,ip], but in fact one can check that all such choices of j give exactly the same expression in (5.2). As before, we identify 1/c(G) with ∑dQd(x−). Set c^{\prime}(k)=c(k)\bigr{\rvert}_{c(\mathcal{G})\to 1}.
Lemma 5.10**.**
Let n be even, and fix G(f)∈GOG. Suppose y=y′×12rSp or y=12rSp×y′ where y′∈In is Sp-vexillary, and that codimOGXy=∣DSp(y)∣. Then [OGXy] is represented by Pλ(c(1),…,c(ks)), and [OGXy(G(f))] is represented by Pλ(c′(1),…,c′(ks)), where λ=shSp(y). If moreover y∈In\textscfpf, then SySp(x)=Pλ(c′(1),…,c′(ks)) as polynomials.
Proof.
Analogous to the proofs of Lemma 5.6 and Theorem 5.7, replacing Lemma 4.22 by Lemma 4.24.
∎
If y′∈In\textscfpf is vexillary, then it is Sp-vexillary and the hypotheses of Lemma 5.10 hold for y=y′×12rSp or y=12rSp×y′, because codimOGXy=ℓSp(y)=∣DSp(y)∣ by Lemma 5.4. In §6, we develop some combinatorial tools for working with rank conditions defining OGXy which will reveal OGXy to be a Grassmannian degeneracy locus in more cases than the ones considered here. This will give Pfaffian formulas under hypotheses more general and less awkward than those of Lemma 5.10 (Theorem 6.13).
Theorem 5.11**.**
For z∈In\textscfpf, the back-stable involution Schubert polynomial SzSp represents the class [OGXz].
Proof.
Analogous to the proof of Theorem 5.9, using Lemma 5.10.
∎
From Theorems 5.9 and 5.11 we obtain geometric interpretations of the involution Stanley symmetric functions FyO=SyO∣x+→0 and FzSp=SzSp∣x+→0.
Theorem 5.12**.**
Fix E∙∈Fl(n). For y∈In and z∈In\textscfpf, the symmetric functions 2cyc(y)FyO and FzSp represent the classes in H∗(LG(2n)) and H∗(OG(2n)) of involution graph Schubert varieties: the closures of, respectively,
[TABLE]
and
[TABLE]
Corollary 5.13**.**
2cyc(y)FyO is Schur Q positive, and FzSp is Schur P positive.
Proof.
Pragacz [26] showed that the classes of the Schubert varieties in H∗(LG(n))≃ΓQ/(Qn+1,Qn+2,…) are represented by Schur Q-functions, which implies that f∈ΓQ represents the class of a variety if and only if f is Schur Q positive modulo the ideal (Qn+1,Qn+2,…). Since 2cyc(y)FyO=2cyc(y×1m)Fy×1mO for all m, Theorem 5.12 implies that 2cyc(y)FyO is Schur Q positive modulo ⋂m(Qn+m+1,Qn+m+2,…)=0. The same argument works for FzSp.
∎
6. I-Grassmannian and Sp-vexillary involutions
Definition 6.1**.**
An involution of the form (ϕ1,m+1)(ϕ2,m+2)⋯(ϕk,m+k) where ϕ1<⋯<ϕk<m is called I-Grassmannian, or m-I-Grassmannian if we wish to specify m.
An involution y∈In is m-I-Grassmannian if and only if Ess(DO(y))⊆{m}×N.
Example 6.3**.**
y=(1,5)(2,6)(4,7) is 4-I-Grassmannian,
[TABLE]
and shO(y)=(4,2,1) and Ess(y)={(4,2),(4,4)}.
In general, if y=(ϕ1,m+1)⋯(ϕk,m+k) is m-I-Grassmannian then the parts of shO(y) are the nonzero column lengths of DO(y), so shO(y)i=m−ϕi+1 for i∈[k].
Theorem 6.4**.**
Fix E∙∈Fl(n). If y∈In is m-I-Grassmannian, then LGXy(E∙)⊆LG(2n) is a Schubert variety whose class in H∗(LG(2n)) is represented by QshO(y).
Proof.
Write y=(ϕ1,m+1)⋯(ϕk,m+k) where ϕ1<⋯<ϕk≤m≤m+k≤n. Considering the form of DO(y), it is clear that Ess(DO(y))⊆{(m,ϕp):p∈[k]} and that ϕp−rky[m][ϕp]=p for p∈[k]. Lemma 5.5 and Lemma 5.4(c) therefore imply that LGXy(E∙) is the closure of
[TABLE]
(To be precise, in (14) we are imposing all rank conditions coming from row n of DO(y), not just those coming from the essential set. However, Lemma 5.5 shows that after taking closures, we get the same variety whether we impose rank conditions for all (i,j) or only the essential ones. Hence, the same is true if any intermediate set of rank conditions is imposed, as are we doing here.)
The closure of (14) is also the Schubert variety in LG(2n) labeled by the strict partition (m+1−ϕ1,…,m+1−ϕk)=shO(y) with respect to the isotropic flag
[TABLE]
and its class is represented by QshO(y) [26, §6].
∎
Corollary 6.5**.**
If y is I-Grassmannian, then FyO=PshO(y).
Proof.
Say y is r-I-Grassmannian, and set y′=1n−r×y for some fixed n≥r, so that y′ is n-Grassmannian and Fy′O=FyO. Theorem 6.4 and Theorem 5.12 imply that 2cyc(y′)Fy′O equals QshO(y′)=2ℓ(shO(y′))PshO(y′) modulo the ideal (Qn+1,Qn+2,…), given that they both represent the same class in H∗(LG(2n)). Since cyc(y′), shO(y′), and Fy′O are all independent of n, letting n→∞ shows that 2cyc(y)FyO=2ℓ(shO(y))PshO(y), which is equivalent to the claimed formula.
∎
The formula of Corollary 6.5 was obtained earlier in [12], where it was used as a base case for a recurrence of the form FyO=∑zFzO; via this recurrence one can compute the Schur P expansion of FyO, and in particular deduce that it is Schur P positive. Theorem 6.4 provides a new and geometrically natural reason to consider I-Grassmannian involutions.
Recall that w∈Sn is vexillary if it avoids the permutation pattern 2143. Stanley showed that Fw is a single Schur function sλ if and only if w is vexillary [27]. In [12] it was shown that 2cyc(y)FyO=Qμ for some strict partition μ if and only if y is vexillary, and our results recover one of these implications.
Theorem 6.6**.**
If y is vexillary, then 2cyc(y)FyO=QshO(y).
Proof.
Apply Theorem 5.7, setting the variables x+ to zero.
∎
We now turn to fixed-point-free involutions and OG(2n). The obvious guess to define an “fpf-I-Grassmannian” involution z would be to require that Ess(DSp(z))⊆{m}×N. However, this condition is too restrictive: it does imply that OGXz(E∙) is a Schubert variety just as in Theorem 6.4, but one cannot obtain a Schubert variety for every strict partition this way. For instance, no such z has shSp(z)=(3,1). The problem is that seemingly different sets of rank conditions on a skew-symmetric map can turn out to be equivalent in ways not seen with symmetric maps. For instance, if z=(1,3)(2,5)(4,6) then
[TABLE]
Therefore G(f)∈OGXz(E∙) if and only if rk(f:E2→E1∗)≤0 and rk(f:E4→E2∗)≤1, corresponding to the two elements of Ess(DSp(z)). But rk(f:E4→E2∗)≤1 implies rk(f:E2→E2∗)≤1, which implies rk(f:E2→E2∗)≤0 because a skew-symmetric map must have even rank, which in turn implies rk(f:E2→E1∗)≤0. Thus, OGXz(E∙) is actually defined by the single rank condition rk(f:E4→E2∗)≤1, so is a Schubert variety in LG(12).
Suppose z∈I2r\textscfpf. Let z=(a1,b1)⋯(ar,br) be the disjoint cycle decomposition of z, where ai<bi for each i. Define involutions dearcR(z) and dearcL(z) as the product of (ai,bi) over all i such that, respectively,
•
ai<bj<bi for some j;
•
ai<aj<bi for some j.
Example 6.8**.**
If z=(1,3)(2,5)(4,6), then dearcR(z)=(2,5)(4,6) and dearcL(z)=(1,3)(2,5). Drawing z as a perfect matching, one obtains dearcR(z) by removing all arcs which do not have the right endpoint of another arc underneath them, and similarly for dearcL(z) changing “right” to “left”:
z=dearcR(z)=dearcL(z)=
Recall that for general y∈In, we define OGXy as the closure in OG(2n)×Fl(n) of
[TABLE]
As a technical crutch in the next few lemmas, let us also define OGXy≤ as the closure in OG(2n)×Fl(n) of
[TABLE]
Lemma 6.9**.**
Let y∈In, and let (a,b) be a cycle in y with a<b and such that a<i<b implies y(i)<a. Set y′=y(a,b). Then OGXy=OGXy′.
Proof.
We prove the stronger fact that DSp(y′)=DSp(y) and rky[i][j]′=rky[i][j] for (i,j)∈DSp(y). Observe that D(y) and D(y′) agree outside of the square [a,b]×[a,b], and contain the following respective configurations:
\circ$$\times$$\times$$a$$b$$a$$b
\times$$\times$$a$$b$$a$$b
Here × denotes a point (i,w(i)) and ∘ a point in D(w), where w is y or y′ as appropriate. The shaded regions contain no × and hence no ∘, from which it follows that D(y′)=D(y)∖{(a,a)}, so DSp(y′)=DSp(y). Also, if (i,j)∈DSp(y) then (i,j)∈/[a,b)×[a,b), which in turn implies rky[i][j]=rky[i][j]′.
∎
Lemma 6.10**.**
Let y∈In, and let (a,b) be a cycle in y with a<b. Assume that a<i<b implies y(i)>b and that rky[b][b] is even. Set y′=y(a,b). Then
(a)
The sets Ess(DSp(y′)) and Ess(DSp(y)) agree outside [a,b]×[a,b].
2. (b)
rky[i][j]=rky[i][j]′* if (i,j)∈Ess(DSp(y′)) or (i,j)∈/[a,b)×[a,b).*
3. (c)
OGXy′≤⊆OGXy≤.
Proof.
D(y) and D(y′) agree outside of the square [a,b]×[a,b], and contain the following respective configurations:
[TABLE]
From (15) one can see that if (i,j)∈/[a,b)×[a,b), then rky[i][j]′=rky[i][j]. Also, the only cell (i,j)∈Ess(DSp(y′)) which could also be in [a,b]×[a,b] is (b,b−1), and (15) again shows that rky[b][b−1]′=rky[b][b−1]. This proves parts (a) and (b) of the lemma. As for part (c), we consider three cases:
(i)
Suppose b=a+1. Then Ess(DSp(y′))=Ess(DSp(y)), and OGXy≤=OGXy′≤ since these two varieties are defined by the same rank conditions.
2. (ii)
Suppose b>a+1 and (b+1,b−1)∈/DSp(y). Then (b−1,b−2)∈Ess(DSp(y)) and Ess(DSp(y′))=Ess(DSp(y))∖{(b−1,b−2)}∪{(b,b−1)}. Moreover, Ess(DSp(y))∩[a,b]×[a,b]={(b−1,b−2)} and Ess(DSp(y′))∩[a,b]×[a,b]={(b,b−1)}. Thus, OGXy′≤ is defined by the same rank conditions as OGXy≤ except that the condition
[TABLE]
is replaced by
[TABLE]
We must show that (17) implies (16). Suppose that (17) holds. Then
[TABLE]
By assumption, 1+rky[b−1][b−1]=−1+rky[b][b] is odd. On the other hand, rk(f:Eb−1→Eb−1∗) is even because f is skew-symmetric. It follows that one of the inequalities (18) and (19) is strict, so that (16) holds.
3. (iii)
Finally, suppose b>a+1 and (b+1,b−1)∈DSp(y). Now Ess(DSp(y′))=Ess(DSp(y))∖{(b−1,b−2)}, and no element of Ess(DSp(y′)) lies in [a,b)×[a,b), so OGXy≤ is defined by the same rank conditions as OGXy′≤ together with the extra condition (16). We must see that this extra condition is actually implied by those defining OGXy′≤.
If (i,j),(i′,j′)∈D(y) satisfy (i′,j′)∈{(i+1,j),(i,j+1)}, then rky[i][j]=rky[i′][j′], so rk(f:Ej′→Ei′∗)≤rky[i′][j′] implies rk(f:Ej→Ei∗)≤rky[i][j]. Thus, if the rank condition rk(f:Ej→Ei∗)≤rky[i][j] holds for every (i,j)∈Ess(DSp(y)), then it holds for every (i,j)∈DSp(y) by induction. In particular, (b,b−1)∈DSp(y′), so (17) holds on OGXy′≤, which implies (16) by case (ii). ∎
Theorem 6.11**.**
Suppose z∈In\textscfpf. Then OGXz=OGXdearcL(z)=OGXdearcR(z).
Proof.
Since dearcL(z) is obtained from z by a sequence of transformations y⇝y′ as in the statement of Lemma 6.9, that lemma implies that OGXz=OGXdearcL(z).
As for dearcR(z), let C be the set of cycles (a,b) of z such that a<b and a<i<b implies z(i)>b. The cycles in C are necessarily non-nesting and non-crossing in the sense that C={(a1,b1),…,(ak,bk)} where
[TABLE]
Set zp=z(a1,b1)⋯(ap,bp) for 0≤p≤k, so z0=z and zk=dearcR(z). The permutation matrices z[bp][bp] and z[bp][bp]p−1 are the same given that a1,b1,…,ap−1,bp−1>bp; and, since z is fixed-point-free, rkz[bp][bp]p−1=rkz[bp][bp] is even. Each transformation zp−1⇝zp is therefore of the form considered in Lemma 6.10(c), so that lemma implies OGXzp≤⊆OGXzp−1≤ for p=1,…,k, hence OGXdearcR(z)≤⊆OGXz≤.
Now we claim that if (i,j)∈Ess(DSp(dearcR(z))), then rkdearcR(z)[i][j]=rkz[i][j]. If (i,j) is not in any square [ap,bp)×[ap,bp), this is clear from Lemma 6.10(b). Suppose (i,j)∈[ap,bp)×[ap,bp). The squares [ap,bp)×[ap,bp) are disjoint by (20), so if q=p then (i,j)∈/[aq,bq)×[aq,bq), hence rkz[i][j]q−1=rkz[i][j]q by Lemma 6.10(b). But given that (i,j)∈/[aq,bq)×[aq,bq) for q>p, Lemma 6.10(a) shows that (i,j)∈Ess(DSp(zp−1)). Thus rkz[i][j]p−1=rkz[i][j]p by Lemma 6.10(b).
Suppose (G(f),E∙)∈OGXz, so rk(EjfEi∗)=rkz[i][j] for i,j∈[n]. The previous paragraph then implies that rk(EjfEi∗)=rkdearcR(z)[i][j] for (i,j)∈Ess(DSp(dearcR(z))), so (G(f),E∙)∈OGXdearcR(z). We now see
[TABLE]
As OGXz=OGXz≤ by the proof of Lemma 5.4(b), taking closures proves the theorem.
∎
Lemma 6.12**.**
If z∈In\textscfpf, then shSp(z)=shSp(dearcR(z))=shSp(dearcL(z)).
Proof.
Since DSp(z)=DSp(dearcL(z)) as per the proof of Lemma 6.9, we get shSp(z)=shSp(dearcL(z)). As for dearcR(z), suppose y and y′ are related as in Lemma 6.10. The diagrams (15) show that ciSp(y)=ciSp(y′) if i∈/[a,b], while for some r,
[TABLE]
So, shSp(y)=shSp(y′), which proves the lemma because dearcR(z) is obtained from z by a sequence of transformations of the form y⇝y′, as per the proof of Lemma 6.11.
∎
We can now state the Pfaffian formulas for SzSp and SzSp promised in §5.2. Recall that y∈In if it is vexillary and Ess(DSp(z)) is a chain under
↗
≤
.
Theorem 6.13**.**
Let z∈In\textscfpf. Suppose y∈{z,dearcR(z),dearcL(z)} is Sp-vexillary. Let \operatorname{Ess}(D^{\operatorname{Sp}}(y))=\{(i_{1},j_{1})\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-3.05mm}\hbox{<}}\cdots\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-3.05mm}\hbox{<}}(i_{s},j_{s})\}, and let kp=jp−rky[ip][jp] for p∈[s] and k0=0. Set λ=shSp(z). For k∈[ks], set
[TABLE]
where kp−1<k≤kp, and c^{\prime}(k)=c(k)\bigr{\rvert}_{x_{-}\to 0}. Then SzSp=Pλ(c′(1),…,c′(ks)) and SzSp=Pλ(c(1),…,c(ks)).
Proof.
Fix G(f)∈OG(2n). By Theorem 6.11 we have OGXy=OGXz. Also, shSp(y)=shSp(z) by Lemma 6.12. Thus, codimOGXy=ℓSp(z)=∣shSp(z)∣=∣shSp(y)∣, so Lemma 5.10 shows that PshSp(y)(c′(1),…,c′(ks)) represents the class [OGXy(G(f))]=[OGXz(G(f))]. By Theorem 4.6, SzSp also represents [OGXz(G(f))].
All of these statements still hold when z is replaced by z×12rSp for any r and y is replaced by y×12r or y×12rSp accordingly. These replacements do not change PshSp(y)(c′(1),…,c′(ks)) or SzSp, so both of these polynomials represent the classes [OGXz×12rSp(G(f))] for all r, hence are equal by Lemma 2.11. To conclude that SzSp=Pλ(c(1),…,c(ks)), apply a limiting argument as in the proof of Theorem 3.9. ∎
The next definition gives an important class of fixed-point-free involutions to which Theorem 6.13 applies.
Definition 6.14**.**
A fixed-point-free involution z is m-fpf-I-Grassmannian if dearcR(z) is m-I-Grassmannian, or simply fpf-I-Grassmannian.
Example 6.15**.**
Let z=(1,3)(2,5)(4,6). Then dearcR(z)=(2,5)(4,6), which is 4-I-Grassmannian with shifted shape (3,1), so z is fpf-I-Grassmannian with fpf shifted shape (2). Note that z itself is not I-Grassmannian, nor is dearcL(z)=(1,3)(2,5).
Theorem 6.16**.**
Fix E∙∈Fl(n). If z∈In\textscfpf is fpf-I-Grassmannian, then OGXz(E∙)⊆OG(2n) is a Schubert variety whose class in H∗(OG(2n)) is represented by PshSp(z).
Proof.
Write dearcR(z)=(ϕ1,m+1)⋯(ϕk,m+k) where ϕ1<⋯<ϕk≤m≤m+k≤n. As in the proof of Theorem 6.4, we compute Ess(DSp(dearcR(z))) and find that OGXz(E∙) is the closure of
[TABLE]
now applying Theorem 6.11 as well. This is a Schubert variety in OG(2n) indexed by the strict partition (n−ϕ1,…,n−ϕk)=sh\textscfpf(z), and its class is represented by Psh\textscfpf(z) [26, §6].
∎
Corollary 6.17**.**
If z is fpf-I-Grassmannian, then FzSp=Psh\textscfpf(z).
Proof.
Follows from Theorem 6.16 and Theorem 4.6 as in the proof of Corollary 6.5.
∎
The formula of Corollary 6.17 was obtained earlier in [15], where it was used as a base case for a recurrence of the form FzSp=∑z′Fz′Sp; via this recurrence one can compute the Schur P expansion of FzSp, and in particular deduce that it is Schur P positive. Theorems 6.16 and 6.11 provide natural geometric reasons to consider fpf-I-Grassmannian involutions and the dearc operations.
Theorem 6.18**.**
If z, dearcL(z), or dearcR(z) is Sp-vexillary, then FzSp=PshSp(z).
Proof.
Apply Theorem 6.13, setting the variables in x+ to zero.
∎
The converse of Theorem 6.18 is false: z=(1,3)(2,4)(5,7)(6,8) has FzSp=P2, but none of z, dearcL(z), or dearcR(z) are Sp-vexillary or even vexillary. In [15], the fixed-point-free involutions z such that FzSp=Pμ for some strict partition μ were characterized as those avoiding a finite list of patterns.
7. Tableau formulas
In this section we use Proposition 4.14 and Theorem 5.7 to give tableau formulas for SyO when y∈In is vexillary.
Definition 7.1**.**
An essential path for a vexillary involution y∈In is a sequence of points (a1,b1),(a2,b2),…,(an+1,bn+1) in [0,n]×[0,n] such that
•
(a_{r+1},b_{r+1})\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-3.05mm}\hbox{<}}(a_{r},b_{r}) for 1≤r≤n;
•
a1=b1;
•
Each (i,j)∈Ess(DO(y)) occurs as some (ar,br).
Note that an essential path necessarily ends at (n,0). Thinking of [0,n]×[0,n] as a graph with edges between horizontally or vertically adjacent lattice points, an essential path is a directed path from the diagonal to the southwest corner (n,0) which moves south or west at each step, and hits every point in Ess(DO(y)).
Example 7.2**.**
Here are two possible essential paths for y=(1,5)(2,4). Cells in DO(y) are marked by circles and cells in Ess(DO(y)) by shaded circles, while × indicates a point (i,y(i)); the lower left corner is (5,0):
\times$$\times$$\times$$\times$$\times
Suppose P is an essential path for some y∈In. Define an (n+1)-tuple xP by x1P=0 and
[TABLE]
If P is the solid red path starting at (2,2) in Example 7.2, xP is (0,x3,−x2,x4,x5,−x1). If P is the dashed blue path starting at (3,3), then xP is (0,−x3,x4,−x2,−x1,x5).
Recall the multiparameter Schur Q-function Qλ(x;t) from Definition 4.12.
Theorem 7.3**.**
Suppose y∈In is vexillary with involution shape shO(y)=λ, and P is an essential path for y starting at diagonal position (j,j). Then 2cyc(y)SyO=Qλ(x−∞⋯j;−xP).
Proof.
Let \mathcal{P}=\{(a_{n+1},b_{n+1})\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-3.05mm}\hbox{<}}\cdots\vbox{\hbox{\hskip 1.42262pt\rotatebox{-15.0}{\scriptscriptstyle\nearrow}}\vspace{-3.05mm}\hbox{<}}(a_{1},b_{1})\}, and set E(r)=Ebr⊕Ear⊥ for each r∈[n+1], where E∙ is the flag of tautological bundles over Fl(n). The chain
[TABLE]
is then a complete isotropic flag. As per Lemma 5.5, LGXy is a Lagrangian Grassmannian degeneracy locus defined by conditions on intersections with certain of the bundles E(r), namely those for which (ar,br)∈Ess(DO(y)). Accordingly, by Proposition 4.14[LGXy] is represented by Qλ(x;t′) where t1′=0 and tr+1′=c1(E(r)/E(r+1)), and Qd(x) is identified with (c(G)c(E(1))1)d.
Under our identifications from §4.2, 1/c(G) is represented by ∑d≥0Qd(x−)=∏i=1∞1−x−i1+x−i while
[TABLE]
where j=a1=b1. Thus, (c(G)c(E(1))1)d is identified with Qd(x−∞⋯j). Also,
[TABLE]
It follows that Qλ(x−∞⋯j;−xP) represents the class [LGXy]. We claim that it represents the classes [LGXy×1m] for all m. Let P+ be the path P with a step from (n,0) to (n+1,0) appended at the end. Then P+ is an essential path for y×1 since Ess(DO(y×1))=Ess(DO(y)), so Qλ(x−∞⋯j;−xP+) represents [LGXy×1] by what we just showed. Definition 4.12 makes clear that Qλ(x;t) is independent of tr for r>λ1; since λ1<n, we see that Qλ(x−∞⋯j;−xP)=Qλ(x−∞⋯j;−xP+) also represents [LGXy×1], and by induction every class [LGXy×1m]. Since 2cyc(y)SyO also represents these classes by Theorem 5.9, Lemma 2.11 forces Qλ(x−∞⋯j;−xP)=2cyc(y)SyO.
∎
Theorem 7.3 writes 2cyc(y)SyO as a sum of products of expressions xi±xj, each product indexed by a tableau. It would also be interesting to express 2cyc(y)SyO as a sum of honest monomials associated to tableaux. In the simple case that Ess(DO(y)) has one element, which implies y=(a,m+1)(a+1,m+2)⋯(a+k−1,m+k) where a+k−1≤m, one can show that
[TABLE]
where T runs over marked shifted semistandard tableaux on (−∞,m] in which every entry exceeding k is primed.
Bibliography32
The reference list from the paper itself. Each links out to its DOI / PubMed record.
1[1] D. Anderson. Double Schubert polynomials and double Schubert varieties. https://people.math.osu.edu/anderson.2804/papers/geomschpolyn.pdf, 2007.
2[2] D. Anderson and W. Fulton. Degeneracy loci, Pfaffians, and vexillary signed permutations in types B, C, and D. ar Xiv:1210.2066 v 1, 2012.
3[3] D. Anderson and W. Fulton. Chern class formulas for classical-type degeneracy loci. ar Xiv:1504.03615 v 2, 2015.
4[4] E. Bagno and Y. Cherniavsky. Congruence B 𝐵 B -orbits and the Bruhat poset of involutions of the symmetric group. Discrete Math. , 312:1289–1299, 2012.
5[5] N. Bergeron and S. Billey. RC-graphs and Schubert polynomials. Experiment. Math. , 2(4):257–269, 1993.
6[6] S. Billey, W. Jockusch, and R. P. Stanley. Some combinatorial properties of Schubert polynomials. J. Algebraic Combin. , 2:345–374, 1993.
7[7] Armand Borel. Sur la cohomologie des espaces fibrés principaux et des espaces homogènes de groupes de Lie compacts. Annals of Mathematics , 57:115–207, 1953.
8[8] T. Bröcker and T. tom Dieck. Representations of Compact Lie Groups . Springer-Verlag, 1985.