Virtual classes and virtual motives of Quot schemes on threefolds

TL;DR

This paper constructs and analyzes virtual classes and motives for Quot schemes on threefolds, leading to new higher rank Donaldson-Thomas invariants and advancing enumerative geometry in Calabi-Yau contexts.

Contribution

It introduces a symmetric obstruction theory for Quot schemes on threefolds and constructs virtual motives, expanding the framework for higher rank Donaldson-Thomas invariants.

Findings

Constructed symmetric obstruction theory on Quot schemes

Developed virtual motives for Quot schemes on threefolds

Computed motivic partition functions and new invariants

Abstract

For a simple, rigid vector bundle $F$ on a Calabi-Yau $3$-fold $Y$, we construct a symmetric obstruction theory on the Quot scheme $\textrm{Quot}_Y(F,n)$, and we solve the associated enumerative theory. We discuss the case of other $3$-folds. Exploiting the critical structure on $\textrm{Quot}_{\mathbb A^3}(\mathscr O^r,n)$, we construct a virtual motive (in the sense of Behrend-Bryan-Szendr\H{o}i) for $\textrm{Quot}_Y(F,n)$ for an arbitrary vector bundle $F$ on a smooth $3$-fold $Y$. We compute the associated motivic partition function. We obtain new examples of higher rank (motivic) Donaldson-Thomas invariants.

Equations

$$\operatorname{Quot}_{Y}(F,n)=\Set{F\twoheadrightarrow Q\,\big{missing}}{\,\dim(Q)=0,\,\chi(Q)=n}$$

$${\mathsf{DT}}_F(q)=\sum _{n\ge 0}{q}^n\left({\int }_{[{\mathrm{Quot}}_Y(F,n){]}^{\mathrm{vir}}}1\right)\in \mathbb{Z}[[q]].$$

$${\mathsf{DT}}_F(q)=\mathsf{M}((-1{)}^{r}q{)}^{r\chi (Y)},$$

$$\bigl{[}\operatorname{Quot}_{Y}(F,n)\bigr{]}_{\operatorname{vir}}\in\mathcal{M}_{\mathbb{C}}=K_{0}(\mathrm{Var}_{\mathbb{C}})\bigl{[}\mathbb{L}^{-\frac{1}{2}}\bigr{]}$$

$$\mathsf{Z}_{r}(Y,t)=\sum_{n\geq 0}\,\bigl{[}\operatorname{Quot}_{Y}(F,n)\bigr{]}_{\operatorname{vir}}\cdot t^{n}$$

$$\mathsf{Z}_{r}(Y,(-1)^{r}t)=\operatorname{Exp}\left(\frac{(-1)^{r}t\left[Y\times\mathbb{P}^{r-1}\right]_{\operatorname{vir}}}{\bigl{(}1-(-\mathbb{L}^{-\frac{1}{2}})^{r}t\bigr{)}\bigl{(}1-(-\mathbb{L}^{\frac{1}{2}})^{r}t\bigr{)}}\right).$$

$${\mathsf{DT}}_F(q)=\mathsf{M}(q{)}^{-400}.$$

$$\mathrm{H}_{c}\bigl{(}\operatorname{Quot}_{\mathbb{A}^{3}}(\mathscr{O}^{\oplus r},n),\Phi_{f_{r,n}}\bigr{)}$$

$${\mathrm{Quot}}_Y({\mathcal{I}}_C,n)\subset {\mathrm{Hilb}}_{\chi ({\mathcal{O}}_C)+n}(Y,[C]).$$

$${\mathbb{L}}_X={\tau }_{\ge -1}{L}_X^{\bullet }\in D^{[-1,0]}(X)$$

$$\mathbb{L}_{X}=\bigl{[}I/I^{2}\xrightarrow{\operatorname{d}}\Omega_{U}\big{|}_{X}\bigr{]}$$

$$\mathcal{M}_{S}=K_{0}(\mathrm{Var}_{S})\bigl{[}\mathbb{L}^{-\frac{1}{2}}\bigr{]}$$

$${\mathcal{M}}_S\times {\mathcal{M}}_{S^{\prime }}\stackrel{⊠}{\to }{\mathcal{M}}_{S\times S^{\prime }}$$

$$(-{\mathbb{L}}^{\frac{1}{2}}{)}^n[X\to S],n\in \mathbb{Z}.$$

$${\tilde{K}}_0^G({\mathrm{Var}}_S)$$

$$[X\to S]\mapsto [X/G\to S/G],$$

$${\mathcal{M}}_S^{\hat{\mu }},$$

$$\chi :{\mathcal{M}}_{\mathbb{C}}^{\hat{\mu }}\to \mathbb{Z},\chi ({\mathbb{L}}^{-\frac{1}{2}})=-1.$$

$$K_0({\mathrm{Var}}_S)$$

$${}^{\diamond }{\sigma }^n(A)={\pi }_{{\mathfrak{S}}_n}(A^{\otimes n})\in {\mathcal{M}}_{S^n/{\mathfrak{S}}_n}.$$

$$\sum _{i=0}^n{\sigma }^i([X]-[Y]){\sigma }^{n-i}[Y]={\sigma }^n[X].$$

$${}^{\diamond }{\sigma }_{\nu }^n(A)={\nu }_{!}{}^{\diamond }{\sigma }^n(A),$$

$${\mathcal{M}}_{\mathbb{C}}[[t]]\tilde{\to }{\mathcal{M}}_{\mathbb{N}}.$$

$$(1+tR[[t]])\times R$$

$$(A(t),m)$$

$$(A(t))_{\operatorname{\diamond}}^{[X]}=1+\sum_{n\geq 0}\sum_{\alpha\vdash n}\pi_{G_{\alpha}}\Biggl{(}\Biggl{[}\prod_{i}X^{\alpha_{i}}\setminus\Delta\Biggr{]}\cdot\prod_{i}A_{i}^{\otimes\alpha_{i}}\Biggr{)}t^{n}.$$

$$(A(t),m)\mapsto A(t{)}^m$$

$$A(t{)}^{[X]}=(C(t){)}_{\diamond }^{[X]}\cdot ((B(t){)}_{\diamond }^{[X]}{)}^{-1},$$

$$A((-\mathbb{L}^{\frac{1}{2}})^{n}t)^{[X]}=A(t)^{[X]}\big{|}_{t\mapsto(-\mathbb{L}^{\frac{1}{2}})^{n}t}.$$

$$\mathrm{Exp}:t{\mathcal{M}}_{\mathbb{C}}[[t]]\tilde{\to }1+t{\mathcal{M}}_{\mathbb{C}}[[t]],$$

$$\mathrm{Exp}(A-B)={}^{\diamond }\mathrm{Exp}(A)\cdot {}^{\diamond }\mathrm{Exp}(B{)}^{-1}.$$

Full text

1. 0 Introduction
2. 1 Preliminaries
3. 2 Obstruction theories on Quot schemes
4. 3 Higher rank Donaldson–Thomas invariants
5. 4 The virtual motive of the Quot scheme

0. Introduction

Overview

The goal of this paper is to show that, for a locally free sheaf $F$ on a complex $3$-fold $Y$, the Quot scheme

[TABLE]

carries a degree [math] virtual fundamental class (under suitable assumptions), as constructed by Behrend–Fantechi [4], as well as a virtual motive in the sense of Behrend–Bryan–Szendrői [3]. Therefore enumerative and motivic invariants can be attached to $\operatorname{Quot}_{Y}(F,n)$. Our results yield new explicit examples of higher rank Donaldson–Thomas invariants and higher rank motivic Donaldson–Thomas invariants of Calabi–Yau $3$-folds.

Our first main result (proved in Theorem 2.5) is the following.

Theorem A**.**

Let $Y$ be a smooth complex projective $3$-fold, $F$ a simple rigid vector bundle on $Y$. Then $\operatorname{Quot}_{Y}(F,n)$ admits a [math]-dimensional perfect obstruction theory in the following situations:

1. (1)

$H^{i}(Y,\mathscr{O}_{Y})=0$ for $i>0$ and $F$ is exceptional; 2. (2)

$Y$ is Calabi–Yau.

In the Calabi–Yau case, the obstruction theory is symmetric.

Under the assumptions of Theorem A, one can see $\operatorname{Quot}_{Y}(F,n)$ as a fine moduli space of simple sheaves (the kernels of the surjections), and form the Donaldson–Thomas partition function

[TABLE]

In the Calabi–Yau case, we deduce (cf. Corollary 3.2) the identity

[TABLE]

where $\mathsf{M}(q)=\prod_{m\geq 1}(1-q^{m})^{-m}$ is the MacMahon function and $r=\operatorname{rk}F$. We conjecture a general formula for $\mathsf{DT}_{F}(q)$, in the case where $(Y,F)$ satisfies (1), in Section 3.3.

To state our second main result, let us fix an arbitrary smooth $3$-fold $Y$, and a vector bundle $F$ on $Y$ of rank $r$. Let $K_{0}(\mathrm{Var}_{\mathbb{C}})$ be the Grothendieck ring of complex varieties, and let $\mathbb{L}=[\mathbb{A}^{1}]$ be the Lefschetz motive. In Section 4 we define motivic weights

[TABLE]

that are virtual motives in the sense of [3], i.e. their Euler characteristic computes the virtual Euler characteristic $\widetilde{\chi}(\operatorname{Quot}_{Y}(F,n))=\chi(\operatorname{Quot}_{Y}(F,n),\nu)\in\mathbb{Z}$ defined by means of Behrend’s microlocal function [2]. We express the generating function

[TABLE]

in terms of the motivic exponential (reviewed in Section 1.2.4). The next result (proven in Theorem 4.12) recovers the calculation [3, Thm. 4.3] by Behrend–Bryan–Szendrői for the Hilbert scheme of points $\operatorname{Hilb}^{n}Y$ if one sets $r=1$.

Theorem B**.**

The motivic partition function (0.2) satisfies

[TABLE]

If $F$ is a simple, rigid vector bundle on a Calabi–Yau $3$-fold $Y$, the coefficients of the series (0.2) refine the enumerative Donaldson–Thomas invariants encoded in (0.1). Thus Theorem B explicitly computes generating functions of higher rank motivic Donaldson–Thomas invariants. As an example, consider a stable arithmetically Cohen–Macaulay rank $2$ bundle $F$ on a general quintic $Y\subset\mathbb{P}^{4}$ (cf. Example 3.3). Then $F$ is rigid, and Theorem B yields (up to a sign) a refinement of the enumerative formula

[TABLE]

Cohomological DT theory

It is proven in [1, Thm. 2.6] that $\operatorname{Quot}_{\mathbb{A}^{3}}(\mathscr{O}^{\oplus r},n)$ is the critical locus of a regular function $f_{r,n}$ for all $r$ and $n$ (cf. Section 4.1). We observe, using one of the main results of [9], that the compactly supported vanishing cycle cohomology

[TABLE]

is pure, and of Tate type, for all $n$. Moreover, in Section 4.4 we compute, for fixed $r\geq 1$, the generating function of Hodge polynomials of (0.3), cf. Formula (4.11).

Related work in the rank $1$ case

The enumerative theory of $\operatorname{Hilb}^{n}Y$ has been solved in [5, 21, 22]. The first breakthrough in motivic Donaldson–Thomas theory was the definition and explicit calculation of the virtual motive of $\operatorname{Hilb}^{n}Y$ on a $3$-fold [3].

Concerning Hilbert schemes of subschemes $Z\subset Y$ with $\dim Z\leq 1$ in a projective $3$-fold $Y$, the contribution of a smooth curve $C\subset Y$, embedded with ideal sheaf $\mathscr{I}_{C}$, is encoded in the Quot scheme

[TABLE]

The $C$-local enumerative Donaldson–Thomas theory was solved in [28, 27], whereas the motivic side was studied by Davison and the author in [10].

Conventions

All schemes are locally of finite type over $\mathbb{C}$. For a scheme $X$, by $\operatorname{D}(X)=\operatorname{D}(\operatorname{QCoh}(X))$ we denote its derived category, and we let $(-)^{\vee}=\operatorname{{\mathbf{R}\mathscr{H}om}}(-,\mathscr{O}_{X})$ be the derived dualising functor. For a torsion free sheaf $E$ on a variety $Y$ we denote by $\operatorname{Ext}^{i}(E,E)_{0}$ the kernel of the trace map $\operatorname{tr}^{i}\colon\operatorname{Ext}^{i}(E,E)\to H^{i}(Y,\mathscr{O}_{Y})$, see [18, Section 10.1] for its construction. A locally free sheaf (or vector bundle) $F$ on a variety is called simple if $\operatorname{Hom}(F,F)=\mathbb{C}$, rigid if $\operatorname{Ext}^{1}(F,F)=0$, exceptional if it is simple and $\operatorname{Ext}^{i}(F,F)=0$ for all $i>0$. A Calabi–Yau $3$-fold is a smooth projective variety $Y$ of dimension $3$, such that $\omega_{Y}\cong\mathscr{O}_{Y}$ and $H^{1}(Y,\mathscr{O}_{Y})=0$.

Acknowledgements

The author wishes to thank Ben Davison, Barbara Fantechi and Martijn Kool for very helpful discussions. We owe a debt of gratitude to Dragos Oprea for generously sharing his insights on the problem. Many thanks to the anonymous referee for spotting several inaccuracies and helping to improve the text. Finally, thanks to SISSA for the great working conditions offered during the completion of this project.

1. Preliminaries

In this section we set the main tools that will be used throughout the paper.

1.1. Obstruction theories

We refer the reader to [4, 5] for more details on obstruction theories and virtual classes. Here we only recall the main definitions.

Let $X$ be a finite type $\mathbb{C}$-scheme, and let $L_{X}^{\bullet}\in\operatorname{D}^{(-\infty,0]}(X)$ be Illusie’s cotangent complex.

Definition 1.1** ([4, Def. 4.4] and [5, Def. 1.10]).**

An obstruction theory on $X$ is a morphism $\phi\colon\mathbb{E}\to L_{X}^{\bullet}$ in $\operatorname{D}(X)$ such that $h^{0}(\phi)$ is an isomorphism and $h^{-1}(\phi)$ is surjective. If $\mathbb{E}$ is perfect of perfect amplitude contained in $[-1,0]$, we say that $\phi$ is perfect. If there exists an isomorphism $\theta\colon\mathbb{E}\,\,\widetilde{\to}\,\,\mathbb{E}^{\vee}[1]$ such that $\theta^{\vee}[1]=\theta$, we say that $\phi$ is symmetric. The virtual dimension of a perfect obstruction theory is the integer $\mathrm{vd}=\operatorname{rk}\mathbb{E}$, i.e. the difference $\operatorname{rk}E^{0}-\operatorname{rk}E^{-1}$ if $\mathbb{E}$ is locally written $[E^{-1}\to E^{0}]$.

Throughout, we let

[TABLE]

be the cut-off at $-1$ of the full cotangent complex. We will only treat perfect obstruction theories, which can be viewed as morphisms $\phi\colon\mathbb{E}\to\mathbb{L}_{X}$. If $X$ is embeddable in a smooth scheme $U$ with ideal sheaf $I\subset\mathscr{O}_{U}$, then one has a canonical isomorphism

[TABLE]

where $\operatorname{d}$ is the exterior derivative.

1.2. Rings of motives and structures on them

Most of the conventions recalled here are taken verbatim from [10, Section 1]. We will need this material (only) in Section 4, so the reader not interested in the motivic part of the paper can safely skip the rest of this section.

Let $S$ be a variety over $\mathbb{C}$, and let $K_{0}(\mathrm{Var}_{S})$ be the Grothendieck ring of $S$-varieties. The ring of motivic weights over $S$ is the ring

[TABLE]

obtained by formally inverting a square root of the Lefschetz motive $\mathbb{L}=[\mathbb{A}^{1}_{S}]$.

A morphism of schemes $f\colon S\to T$ induces, by fibre product, a map of rings $f^{*}\colon\mathcal{M}_{T}\to\mathcal{M}_{S}$, while composition with $f$ gives an $\mathcal{M}_{T}$-linear direct image homomorphism $f_{!}\colon\mathcal{M}_{S}\to\mathcal{M}_{T}$. If $f\colon S\to\operatorname{Spec\,}\mathbb{C}$ is the structure morphism of $S$, we write $\int_{S}$ instead of $f_{!}$ . If $S$ and $S^{\prime}$ are two varieties, the exterior product

[TABLE]

is defined on generators of $K_{0}(\mathrm{Var})$ by sending $(u,v)\mapsto u\times v$ and then extended by linearity.

Definition 1.2**.**

We denote by $S_{0}(\mathrm{Var}_{S})\subset K_{0}(\mathrm{Var}_{S})$ the sub semigroup of effective motives, i.e. the semigroup generated by classes $[X\to S]$ of complex quasi-projective $S$-varieties modulo the scissor relations. Its image in $\mathcal{M}_{S}$ is the sub semigroup $\mathcal{M}_{S}^{\operatorname{eff}}\subset\mathcal{M}_{S}$ consisting of sums of elements of the form

[TABLE]

1.2.1. Equivariant theory

Recall that if $S$ is a variety with a good action by a finite group $G$ (i.e. such that every point of $S$ has an affine $G$-invariant open neighborhood), the quotient $S/G$ exists as a variety.

Definition 1.3**.**

Let $G$ be a finite group, $S$ a variety with good $G$-action. We denote by $\widetilde{K}_{0}^{G}(\mathrm{Var}_{S})$ the abelian group generated by isomorphism classes $[X\to S]$ of $G$-equivariant $S$-varieties with good action, modulo the $G$-equivariant scissor relations. We define the $G$-equivariant Grothendieck group $K_{0}^{G}(\mathrm{Var}_{S})$ by imposing the further relations $[V\to X\to S]=[\mathbb{A}^{r}_{X}]$, whenever $V\to X$ is a $G$-equivariant vector bundle of rank $r$, with $X\to S$ a $G$-equivariant $S$-variety. The element $[\mathbb{A}^{r}_{X}]$ in the right hand side is taken with the $G$-action induced by the trivial action on $\mathbb{A}^{r}$ and the isomorphism $\mathbb{A}^{r}_{X}=\mathbb{A}^{r}\times X$.

There is a natural ring structure on $\widetilde{K}_{0}^{G}(\mathrm{Var}_{S})$ given by taking the diagonal action on $X\times_{S}Y$, for two equivariant $S$-varieties $X\to S$ and $Y\to S$. Inverting a square root of $\mathbb{L}$, one obtains the rings $\widetilde{\mathcal{M}}_{S}^{G}$ and $\mathcal{M}_{S}^{G}$ of $G$-equivariant motivic weights. These rings fit in a commutative diagram

[TABLE]

where the top map $\pi_{G}$ is defined on generators by taking the orbit space,

[TABLE]

and the bottom map is the extension determined by $\mathbb{L}^{\frac{n}{2}}\cdot[X\rightarrow S]\mapsto\mathbb{L}^{\frac{n}{2}}\cdot\pi_{G}[X\rightarrow S]$. The map $\pi_{G}$ does not always extend to $\mathcal{M}_{S}^{G}$. It does if $G$ acts freely on $S$.

A special case of Definition 1.3 yields the monodromic ring of motivic weights

[TABLE]

where $\hat{\mu}=\varprojlim\mu_{n}$ is the procyclic group of roots of unity. We have an Euler characteristic homomorphism

[TABLE]

1.2.2. Lambda ring structures

Let $n>0$ be an integer, and let $\mathfrak{S}_{n}$ be the symmetric group of $n$ elements. By [10, Lemma 1.6], namely the relative version of [3, Lemma 2.4], there exist “$n$-th power” maps fitting in a commutative diagram

[TABLE]

where $S^{n}=S\times\cdots\times S$ carries the natural $\mathfrak{S}_{n}$-action. For $A\in\mathcal{M}_{S}$, define

[TABLE]

The lambda ring operations on $\mathcal{M}_{\mathbb{C}}$ are defined by $\sigma^{n}(A)={}^{\diamond}\!\!\sigma^{n}(A)\in\mathcal{M}_{\mathbb{C}}$ for $A$ effective, and then taking the unique extension to a lambda ring on $\mathcal{M}_{\mathbb{C}}$, determined by the relation

[TABLE]

Note that $\sigma^{n}(-\mathbb{L}^{1/2})=(-\mathbb{L}^{1/2})^{n}$.

If $S$ comes with a commutative associative map $\nu\colon S\times S\to S$, we likewise define

[TABLE]

where we abuse notation by denoting by $\nu$ the map $S^{n}/\mathfrak{S}_{n}\rightarrow S$. As above, using the analogue of the relation (1.4), there is a unique set of lambda ring operators $\sigma_{\nu}^{n}$ agreeing with ${}^{\diamond}\!\!\sigma_{\nu}$ on effective motives.

As a special case, we can consider $(S,\nu)=(\mathbb{N},+)$, viewed as a symmetric monoid in the category of schemes. We obtain operations ${}^{\diamond}\!\!\sigma^{n}$ and $\sigma^{n}$ on $\mathcal{M}_{\mathbb{C}}\llbracket t\rrbracket$ via the isomorphism

[TABLE]

Remark 1.4**.**

The ‘$\diamond$’ decoration will also appear in “preliminary” versions of the power structure on $K_{0}(\mathrm{Var}_{\mathbb{C}})$ (Section 1.2.3) and of the motivic exponential on $\mathcal{M}_{S}$ (Section 1.2.4). Just as in [10], in our formulas from Section 4 we need to prove that we are dealing with effective classes before removing the ‘$\diamond$’ decoration and pass to the classical operations.

1.2.3. Power structures

Definition 1.5** ([16]).**

A power structure on a ring $R$ is a map

[TABLE]

satisfying the following conditions:

1. (1)

$A(t)^{0}=1$, 2. (2)

$A(t)^{1}=A(t)$, 3. (3)

$(A(t)\cdot B(t))^{m}=A(t)^{m}\cdot B(t)^{m}$, 4. (4)

$A(t)^{m+m^{\prime}}=A(t)^{m}\cdot A(t)^{m^{\prime}}$, 5. (5)

$A(t)^{mm^{\prime}}=(A(t)^{m})^{m^{\prime}}$, 6. (6)

$(1+t)^{m}=1+mt+O(t^{2})$, 7. (7)

A(t)^{m}\big{|}_{t\to t^{e}}=A(t^{e})^{m}.

Throughout we use the following:

Notation 1.6**.**

Partitions $\alpha\vdash n$ are written as $\alpha=(1^{\alpha_{1}}\cdots i^{\alpha_{i}}\cdots s^{\alpha_{s}})$, meaning that there are $\alpha_{i}$ parts of size $i$. In particular we recover $n=\sum_{i}i\alpha_{i}$. The automorphism group of $\alpha$ is the product of symmetric groups $G_{\alpha}=\prod_{i}\mathfrak{S}_{\alpha_{i}}$.

If $X$ is a variety and $A(t)=1+\sum_{n>0}A_{n}t^{n}\in K_{0}(\mathrm{Var}_{\mathbb{C}})\llbracket t\rrbracket$ is a power series, we define

[TABLE]

In the above formula, $\Delta\subset\prod_{i}X^{\alpha_{i}}$ is the “big diagonal” (the locus in the product where at least two entries are equal), and the product in big round brackets is a $G_{\alpha}$-equivariant motive, thanks to the power map (1.3). Gusein-Zade, Luengo and Melle-Hernández have proved [16, Thm. 2] that there is a unique power structure

[TABLE]

on $K_{0}(\mathrm{Var}_{\mathbb{C}})$ for which the restriction to the case where all $A_{i}$ and $m$ are effective is given by the formula (1.6). Since we always consider effective exponents when taking powers, we just recall the recipe for dealing with general $A(t)$ and effective exponent $[X]$. First, note that for any such $A(t)$ there is an effective $B(t)$ such that $A(t)\cdot B(t)=C(t)$ is effective. Then we have

[TABLE]

where both factors in the right hand side are defined via (1.6).

As noted in [3], there is an extension of the power structure to $\mathcal{M}_{\mathbb{C}}$ uniquely determined by the substitution rules

[TABLE]

1.2.4. Motivic Exponential

The plethystic, or motivic exponential is a group isomorphism

[TABLE]

converting sums into products. First, define ${}^{\diamond}\!\!\operatorname{Exp}=\sum_{n\geq 0}{}^{\diamond}\!\!\sigma^{n}$, where ${}^{\diamond}\!\!\sigma^{n}$ are (up to the identification (1.5)) the lambda ring operations relative to the monoid $(\mathbb{N},+)$. Then if $A$, $B\in\mathcal{M}^{\operatorname{eff}}_{\mathbb{N}}$ are effective classes, define

[TABLE]

If $(S,\nu\colon S\times S\rightarrow S)$ is a commutative monoid in the category of schemes, with a submonoid $S_{+}\subset S$ such that the induced map $\coprod_{n\geq 1}S_{+}^{\times n}\rightarrow S$ is of finite type, we similarly define

[TABLE]

and for $A$, $B\in\mathcal{M}^{\operatorname{eff}}_{S}$ two effective classes, we set

[TABLE]

1.2.5. Motives over symmetric products

The machinery described so far will be applied in Section 4 to the following situation. For a variety $V$, we will consider $(\operatorname{Sym}(V),\cup)$, where

[TABLE]

can be viewed as a monoid via the morphism

[TABLE]

sending two [math]-cycles on $V$ to their union. We consider the submonoid $\coprod_{n>0}\operatorname{Sym}^{n}(V)$ to construct the maps ${}^{\diamond}\!\!\operatorname{Exp}_{\cup}$ and $\operatorname{Exp}_{\cup}$ as in Section 1.2.4.

In order to recover a formal power series in $\mathcal{M}_{\mathbb{C}}\llbracket t\rrbracket$ from a relative motive over $\operatorname{Sym}(V)$, we consider the operation

[TABLE]

In other words we take the direct image along the “tautological” map $\#\colon\operatorname{Sym}(V)\rightarrow\mathbb{N}$ which collapses $\operatorname{Sym}^{n}(V)$ onto the point $n$. In the right hand side of (1.7), we use (1.5) to identify relative motivic weights over $\mathbb{N}$ and formal power series with coefficients in $\mathcal{M}_{\mathbb{C}}$.

1.2.6. The virtual motive of a critical locus

For a complex scheme $X$ of finite type over $\mathbb{C}$, recall the virtual Euler characteristic

[TABLE]

where $\nu_{X}\colon X(\mathbb{C})\to\mathbb{Z}$ is Behrend’s canonical constructible function [2].

Definition 1.7** ([3]).**

Let $X$ be a scheme. A motivic class $\xi\in\mathcal{M}_{\mathbb{C}}^{\hat{\mu}}$ such that $\chi(\xi)=\widetilde{\chi}(X)$ is called a virtual motive for $X$. Here $\chi$ is the map (1.2).

Definition 1.8**.**

A scheme $X$ is a critical locus if there exists a smooth scheme $U$ and a regular function $f\colon U\to\mathbb{A}^{1}$ such that $X=Z(\operatorname{d}f)\subset U$.

A critical locus $X=Z(\operatorname{d}f)\subset U$ does not only carry a canonical virtual fundamental class $[X]^{\operatorname{vir}}\in A_{0}X$ (cf. [5]). It also supports a canonical relative motive

[TABLE]

where $\mathsf{MF}$ stands for “Milnor fibre” and $[\phi_{f}]_{X}\in\mathcal{M}_{X}^{\hat{\mu}}$ is the (relative) motivic vanishing cycle class introduced by Denef and Loeser [11]. It can be seen as the virtual motivic analogue of the sheaf of vanishing cycles $\Phi_{f}\in\operatorname{D}^{b}_{c}(X)$.

Notation 1.9**.**

If $X=Z(\operatorname{d}f)\subset U$ is a critical locus, $i\colon Z\hookrightarrow X$ is a subscheme and $X\to Y$ is a morphism, we define

[TABLE]

When $X\to Y=\operatorname{Spec\,}\mathbb{C}$ is the structure morphism, we simply write $[Z]_{\operatorname{vir}}$.

Set $[\phi_{f}]=\int_{X}[\phi_{f}]_{X}$. Then, by [3, Prop. 2.16], the motivic weight

[TABLE]

is a virtual motive for $X$, in the sense of Definition 1.7.

Example 1.10**.**

When $f=0$, we have $X=U$ and $[\phi_{f}]=-[X]$, thus

[TABLE]

This is the motivic analogue of the relation

[TABLE]

Assume $X$ is proper. Applying $\chi$ (resp. the degree map) to the first (resp. the second) identity yields the virtual Euler characteristic $\widetilde{\chi}(X)=(-1)^{\dim X}\chi(X)$.

Example 1.11**.**

More generally, if $X=Z(\operatorname{d}f)$ is proper, Behrend’s theorem [2] reads

[TABLE]

expressing the relation between the virtual class of $X$ and its virtual motive.

2. Obstruction theories on Quot schemes

For a coherent sheaf $F$ on a variety $Y$, and an integer $n\geq 0$, the Quot scheme

[TABLE]

parameterises short exact sequences

[TABLE]

where $Q$ is a sheaf supported in dimension [math] with

[TABLE]

Throughout this section, $Y$ denotes a smooth complex projective $3$-fold, and $F$ a locally free sheaf (or vector bundle) of rank $r\geq 1$.

2.1. Tangents and obstructions

For a [math]-dimensional sheaf $Q$, one has $\operatorname{Hom}(F,Q)=\mathbb{C}^{r\cdot\chi(Q)}$. All other Ext groups vanish:

[TABLE]

Given a short exact sequence as in (2.1), these vanishings induce isomorphisms

[TABLE]

Lemma 2.1**.**

Let $Y$ be a smooth projective $3$-fold, $F$ a vector bundle on $Y$, and $F\twoheadrightarrow Q$ a [math]-dimensional quotient with kernel $S$. Then:

• (i)

if $F$ is simple, one has $\operatorname{Hom}(S,F)=\mathbb{C}$,

• (ii)

if $F$ is rigid, one has $\operatorname{Ext}^{1}(S,F)=0$.

Proof.

Applying $\operatorname{Hom}(-,F)$ to the exact sequence $S\hookrightarrow F\twoheadrightarrow Q$ yields

[TABLE]

and the two outer groups vanish by (2.2), so if $F$ is simple we find

[TABLE]

proving (i). The exact sequence above continues as

[TABLE]

where the rightmost group vanishes by (2.2), and the leftmost vanishes if $F$ is rigid (by definition), proving (ii). ∎

Corollary 2.2**.**

In the situation of Lemma 2.1, if $F$ is simple and rigid there is an isomorphism

[TABLE]

and a linear inclusion

[TABLE]

Proof.

Applying $\operatorname{Hom}(S,-)$ to $S\hookrightarrow F\twoheadrightarrow Q$ we obtain

[TABLE]

where the [math] on the right is $\operatorname{Ext}^{1}(S,F)=0$ obtained by Lemma 2.1 (ii). But we have a splitting $\operatorname{Hom}(S,S)=H^{0}(Y,\mathscr{O}_{Y})\oplus\operatorname{Hom}(S,S)_{0}$, thus $i$ is an isomorphism since $\operatorname{Hom}(S,F)\cong\mathbb{C}$ by Lemma 2.1 (i). Hence $u=0$, which implies that $\partial$ is an isomorphism. Finally, the long exact sequence above continues as $0\to\operatorname{Ext}^{1}(S,Q)\to\operatorname{Ext}^{2}(S,S)$, proving the claim. ∎

Fix a short exact sequence as in (2.1), defining a point $x=[F\twoheadrightarrow Q]\in\operatorname{Quot}_{Y}(F)$. Consider the Quot functor $\mathsf{Quot}_{Y}(F)\colon\mathrm{Sch}_{k}^{\textrm{op}}\to\mathrm{Sets}$ and let $\operatorname{Art}_{\mathbb{C}}\subset\mathrm{Sch}_{\mathbb{C}}^{\textrm{op}}$ be the category of local Artinian $\mathbb{C}$-algebras (in other words $\operatorname{Art}_{\mathbb{C}}$ is the category of fat points). As is well-known, the deformation functor

[TABLE]

defined by sending an algebra $A$ to the set of $A$-flat families of quotients restricting to $x$ over the closed fibre, is pro-representable and carries a tangent-obtruction theory $(T_{1},T_{2})$, in the sense of [13], given by the vector spaces $T_{i}=\operatorname{Ext}^{i-1}(S,Q)$. However, this does not give rise to a perfect obstruction theory in the sense of Definition 1.1, for instance because higher Ext groups need not vanish. By Corollary 2.2, the deformation theory of the quotients $F\twoheadrightarrow Q$ is isomorphic to the deformation theory of the kernels $S\subset F$ — see Lemma 2.7 for a precise statement. This allows us to modify the standard obstruction theory (essentially to get a larger obstruction space) by focusing on the kernels of the surjections.

From now on in this section, we make the following:

Assumption 2.3.

The locally free sheaf $F$ on the smooth projective $3$-fold $Y$ is simple and rigid. Moreover, either

• $(\star)$

$H^{i}(Y,\mathscr{O}_{Y})=0$ for $i>0$ and $F$ is exceptional, or

• $(\ast)$

$Y$ is Calabi–Yau.

These are the assumptions of Theorem A.

Recall that a simple coherent sheaf $F$ is exceptional if $\operatorname{Ext}^{i}(F,F)=0$ for all $i>0$. Note that, by our assumption, for any $S\in\operatorname{Coh}(Y)$ we have $\operatorname{Ext}^{i}(S,S)_{0}=\operatorname{Ext}^{i}(S,S)$ for $i=1,2$, and also for $i=3$ in case $(\star)$.

To get a perfect obstruction theory, we will need the following vanishings.

Proposition 2.4**.**

Let $(Y,F)$ satisfy Assumption 2.3. Let $F\twoheadrightarrow Q$ be a [math]-dimensional quotient with kernel $S$. Then

[TABLE]

Proof.

From the splitting $\operatorname{Hom}(S,S)=H^{0}(Y,\mathscr{O}_{Y})\oplus\operatorname{Hom}(S,S)_{0}$ induced by the trace, and the isomorphisms $\operatorname{Hom}(S,S)\cong\operatorname{Hom}(S,F)\cong\mathbb{C}$, we deduce $\operatorname{Hom}(S,S)_{0}=0$. In the Calabi–Yau case $(\ast)$, by Serre duality we obtain the vanishing $\operatorname{Ext}^{3}(S,S)_{0}=0$. In case $(\star)$, consider the surjection $\operatorname{Ext}^{3}(F,S)\twoheadrightarrow\operatorname{Ext}^{3}(S,S)$. By (2.3) we have $\operatorname{Ext}^{3}(F,S)=\operatorname{Ext}^{3}(F,F)=0$, so $\operatorname{Ext}^{3}(S,S)=0$. ∎

2.1.1. Virtual dimension and point-wise symmetry

In the perfect obstruction theory we want to build, the tangent space at $x=[S\hookrightarrow F\twoheadrightarrow Q]\in\operatorname{Quot}_{Y}(F)$ is $\operatorname{Ext}^{1}(S,S)=\operatorname{Hom}(S,Q)$, and the obstruction space is $\operatorname{Ext}^{2}(S,S)$. Its virtual dimension at $x$ would then be

[TABLE]

Note that $\chi(S,S)=\chi(F,F)$. In case $(\star)$, we have $\mathrm{ext}^{3}(S,S)=0$ and $\chi(F,F)=1$, therefore $\mathrm{vd}_{x}=0$. In the Calabi–Yau case, $\mathrm{vd}_{x}=0$ by Serre duality — or, directly, because $\mathrm{ext}^{3}(S,S)=1$ and $\chi(F,F)=0$. So the difference (2.5) is always zero.

In fact, more is true: tangents are always dual to obstructions. This is clear in the Calabi–Yau case. In case $(\star)$, since $F$ is exceptional, one can use both the vanishings $\operatorname{Ext}^{2}(F,S)=\operatorname{Ext}^{3}(F,S)=0$ from (2.3) to obtain an exact sequence

[TABLE]

Dualising, this is an isomorphism

[TABLE]

To sum up, if we manage to produce a perfect obstruction theory with $\operatorname{Ext}^{1}(S,S)$, $\operatorname{Ext}^{2}(S,S)$ as tangents and obstructions, it will be [math]-dimensional and “point-wise symmetric”. However, point-wise symmetry does not imply global symmetry (cf. Definition 1.1), as shown by the case of $\operatorname{Hilb}^{n}Y$ for a $3$-fold $Y$ that is not Calabi–Yau.

2.2. Obstruction theory: construction

Let us shorten $\mathrm{Q}=\operatorname{Quot}_{Y}(F,n)$. Let $p\colon Y\times\mathrm{Q}\to\mathrm{Q}$ and $q\colon Y\times\mathrm{Q}\to Y$ be the projections. Consider the universal exact sequence

[TABLE]

living over $Y\times\mathrm{Q}$. The trace map

[TABLE]

has a canonical splitting, and we denote its kernel by

[TABLE]

The truncated cotangent complex $\mathbb{L}_{Y\times\mathrm{Q}}$ splits as $p^{*}\mathbb{L}_{\mathrm{Q}}\oplus q^{*}\mathbb{L}_{Y}$, so the truncated Atiyah class (cf. [19, Def. 2.6])

[TABLE]

projects onto the factor

[TABLE]

which by the splitting of $\operatorname{tr}_{\SS}$ can be further projected onto

[TABLE]

By Grothendieck duality along the smooth, proper $3$-dimensional morphism $p$, one has

[TABLE]

for $\mathscr{F}\in\operatorname{D}^{b}(Y\times\mathrm{Q})$ and $\mathscr{G}\in\operatorname{D}^{b}(\mathrm{Q})$, where $\omega_{p}=q^{*}\omega_{Y}$ is the relative dualising sheaf. Setting $\mathscr{F}=\operatorname{{\mathbf{R}\mathscr{H}om}}(\SS,\SS)_{0}\otimes\omega_{p}$ and $\mathscr{G}=\mathbb{L}_{\mathrm{Q}}$ in (2.6), we obtain

[TABLE]

which after applying $h^{-2}\circ\mathbf{R}\Gamma$ becomes

[TABLE]

where we have set

[TABLE]

Under the above identifications, the truncated Atiyah class $A(\SS)$ determines a morphism

[TABLE]

We can now give the proof of Theorem A.

Theorem 2.5**.**

If the pair $(Y,F)$ satisfies Assumption 2.3, then $\phi$ is a perfect obstruction theory of virtual dimension [math]. If $Y$ is Calabi–Yau, it is symmetric.

Proof.

The Quot scheme $\mathrm{Q}$ satisfies the assumptions stated in [19, Section 4], namely it is separated and it carries a universal simple sheaf. The latter is just the universal kernel $\SS\in\operatorname{Coh}(Y\times\mathrm{Q})$ viewed as a $\mathrm{Q}$-flat family of simple sheaves on $Y$. Now the argument of [19, Thm. 4.1] applied to $\SS$ proves that $\phi$ is an obstruction theory.

Let us shorten $\mathbb{H}=\operatorname{{\mathbf{R}\mathscr{H}om}}(\SS,\SS)_{0}$. Note that $\mathbb{H}$ is canonically self-dual. The complex $\mathbf{R}p_{*}\mathbb{H}$ is isomorphic in the derived category to a two-term complex of vector bundles $\mathcal{T}^{\bullet}=[\mathcal{T}^{1}\to\mathcal{T}^{2}]$ concentrated in degrees $1$ and $2$. More precisely, as in [19, Lemma. 4.2], the identification $\mathbf{R}p_{*}\mathbb{H}=\mathcal{T}^{\bullet}$ follows from the vanishings

[TABLE]

that we proved in Proposition 2.4. On the other hand, we have

[TABLE]

Therefore $\mathbb{E}$ is perfect in $[-1,0]$, i.e. $\phi$ is perfect.

For any point $x=[S\hookrightarrow F\twoheadrightarrow Q]$, with inclusion $\iota_{x}\colon x\hookrightarrow\mathrm{Q}$, one has

[TABLE]

Therefore we have $\mathrm{vd}=\operatorname{rk}\mathbb{E}=\mathrm{ext}^{1}(S,S)-\mathrm{ext}^{2}(S,S)=0$, as observed in Section 2.1.1.

Let us prove symmetry in the Calabi–Yau case. The argument is standard — see for instance [5] — but we repeat it here for completeness. Any trivialisation $\omega_{Y}\,\widetilde{\to}\,\mathscr{O}_{Y}$ induces, by pullback along $Y\times\mathrm{Q}\to Y$, a trivialisation $\omega_{p}\,\widetilde{\to}\,\mathscr{O}_{Y\times\mathrm{Q}}$, that we can use to construct an isomorphism

[TABLE]

Dualising and shifting the last isomorphism, we get

[TABLE]

where the source is canonically identified with $\mathbb{E}$. The symmetry condition $\theta^{\vee}[1]=\theta$ follows from [5, Lemma 1.23]. ∎

Corollary 2.6**.**

Under the assumptions of Theorem 2.5, the Quot scheme $\operatorname{Quot}_{Y}(F,n)$ has a [math]-dimensional virtual fundamental class

[TABLE]

Since the Quot scheme is proper, we can define Donaldson–Thomas type invariants

[TABLE]

representing the virtual number of points of the Quot scheme. They will be discussed in Section 3.

2.3. Relation with moduli of simple sheaves

In the proof of Theorem 2.5 we viewed the scheme $\operatorname{Quot}_{Y}(F,n)$ as a fine moduli space of simple sheaves via the universal kernel $\SS\subset q^{*}F$. We now prove that $\operatorname{Quot}_{Y}(F,n)$ is indeed an open subscheme of the moduli space

[TABLE]

of simple sheaves with Chern character $v_{n}=\operatorname{ch}(F)-(0,0,0,n)$.

We now recall a classical result from Deformation Theory, stated in the language of tangent-obstruction theories — see e.g. [13, Ch. 6].

Lemma 2.7**.**

Let $\operatorname{D}$, $\operatorname{D}^{\prime}$ be two pro-representable deformation functors and let $(T_{1},T_{2})$, $(T^{\prime}_{1},T^{\prime}_{2})$ be tangent-obstruction theories on them. Let $\eta\colon\operatorname{D}\to\operatorname{D}^{\prime}$ be a morphism inducing an isomorphism $h\colon T_{1}\,\widetilde{\to}\,T_{1}^{\prime}$ and a linear embedding $T_{2}\hookrightarrow T_{2}^{\prime}$. Then $\eta$ is an isomorphism.

Proof.

See [31, Remark 2.3.8] and the surrounding discussion. ∎

Proposition 2.8**.**

Let $F$ be a simple rigid vector bundle on a smooth projective $3$-fold $Y$. Then there is an open immersion $\Psi_{n}\colon\operatorname{Quot}_{Y}(F,n)\to M_{Y,n}$.

Proof.

The map $\Psi_{n}$ takes a surjection to its kernel. This is clearly a morphism, since $\SS$ is flat over $\mathrm{Q}=\operatorname{Quot}_{Y}(F,n)$. It is injective on points (by definition of the Quot functor) and locally of finite type (because the Quot scheme is of finite type over $\mathbb{C}$).

We now show that $\Psi_{n}$ is formally étale. Fix a point $x=[F\twoheadrightarrow Q]\in\operatorname{Quot}_{Y}(F,n)$ with $S=\ker(F\twoheadrightarrow Q)$ and let $s=\Psi_{n}(x)=[S]\in M_{Y,n}$. Consider the deformation functors $\mathsf{Def}_{F\twoheadrightarrow Q}$ and $\mathsf{Def}_{S}$ and their tangent-obstruction theories given respectively by $T_{i}=\operatorname{Ext}^{i-1}(S,Q)$ and $T^{\prime}_{i}=\operatorname{Ext}^{i}(S,S)$ for $i=1,2$. The natural transformation

[TABLE]

taking a surjection to its kernel involves pro-representable functors (note that $\mathsf{Def}_{S}$ is pro-representable because $S$ is simple), and it induces an isomorphism on tangent spaces and an injection on obstruction spaces (cf. Corollary 2.2). Then Lemma 2.7 implies that $\eta$ is an isomorphism of deformation functors. This implies formal étaleness of $\Psi_{n}$ by a direct application of the formal criterion. In a little more detail, consider a square zero extension $\iota\colon T\hookrightarrow\overline{T}$ of fat points, and a commutative diagram

[TABLE]

where $\alpha$ is the unique extension we need to find. Using pro-representability of $\mathsf{Def}_{F\twoheadrightarrow Q}$ and $\mathsf{Def}_{S}$, the condition that $\eta$ is a natural isomorphism translates into a commutative diagram

[TABLE]

where the vertical isomorphisms (composition with $\Psi_{n}$) are precisely the isomorphisms $\eta_{T}$ and $\eta_{\overline{T}}$. Since $\overline{g}\in\operatorname{Hom}_{s}(\overline{T},M_{Y,n})$ lifts to a morphism $\alpha\in\operatorname{Hom}_{x}(\overline{T},\mathrm{Q})$ and both $\alpha\circ\iota$ and $g$ map to $\Psi_{n}\circ g\in\operatorname{Hom}_{s}(T,M_{Y,n})$, they must be equal, for the vertical map on the right is also an isomorphism. Thus $\alpha$ is the required (clearly unique) lift, proving that $\Psi_{n}$ is formally étale.

Thus $\Psi_{n}$ is an injective étale morphism, i.e. an open immersion. ∎

2.4. Symmetry in case $(\star)$

In this section we assume the pair $(Y,F)$ satisfies $(\star)$ and we show that the obstruction theory constructed in Theorem 2.5 in this case becomes symmetric after suitably shrinking the Quot scheme.

The Quot-to-Chow morphism (see [15, Section 6] or [30, Cor. $7.15$] for its construction)

[TABLE]

takes a quotient $[F\twoheadrightarrow Q]$ to the [math]-cycle determined by the set-theoretic support $\operatorname{Supp\,}(Q)\subset Y$, weighted by the length. For any open subscheme $U\subset Y$, the preimage of $\operatorname{Sym}^{n}U\subset\operatorname{Sym}^{n}Y$ under $\sigma_{Y}$ gives an open subscheme

[TABLE]

isomorphic to $\operatorname{Quot}_{U}(F|_{U},n)$. Note that such Quot scheme makes sense, even though $U$ is only quasi-projective, because the support of a family of [math]-dimensional quotients is always proper over the base.

We now consider the diagram

[TABLE]

and form the pullback

[TABLE]

where the identification follows from base change and by $\omega_{\overline{p}}=j^{\ast}\omega_{p}$. Since the inclusions $i$, $j$ and $a$ are open, their pullbacks are underived. Since dualising sheaves are invertible, tensor products $(-)\otimes\omega$ are also underived.

Let us introduce the notation

[TABLE]

Since $\omega_{\pi}=a^{\ast}\omega_{\overline{p}}$, we can write

[TABLE]

where the first identity follows from $\mathbf{R}\pi_{\ast}=\mathbf{R}\overline{p}_{\ast}\circ\mathbf{R}a_{\ast}$ and the second one uses the projection formula along the open immersion $a$. From Equations (2.8) and (2.9), the canonical morphism $\mathscr{O}_{Y\times\mathrm{Q}_{U}}\to\mathbf{R}a_{\ast}\mathscr{O}_{U\times\mathrm{Q}_{U}}$ induces a canonical isomorphism

[TABLE]

To see that $\alpha$ is an isomorphism, it is enough to observe that its cone vanishes. The cone $K$ of $\mathscr{O}_{Y\times\mathrm{Q}_{U}}\to\mathbf{R}a_{\ast}\mathscr{O}_{U\times\mathrm{Q}_{U}}$ is supported on $Z=Y\times\mathrm{Q}_{U}\setminus U\times\mathrm{Q}_{U}$. Now we note that $\mathbb{H}_{Y}\otimes^{\mathbf{L}}K$, which in principle is supported on $Z$, vanishes. To see this, first set $\mathcal{F}=q^{\ast}F$. Then, since $\mathcal{S}|_{Z}=\mathcal{F}|_{Z}$ (because the support of the quotients is now constrained on $U$), we have that $\operatorname{{\mathbf{R}\mathscr{H}om}}(\mathcal{S},\mathcal{S})_{0}|_{Z}=\operatorname{{\mathbf{R}\mathscr{H}om}}(\mathcal{F},\mathcal{F})_{0}|_{Z}$, but $\operatorname{{\mathbf{R}\mathscr{H}om}}(\mathcal{F},\mathcal{F})_{0}=0$ since $F$ is exceptional. Thus $\mathbb{H}_{Y}\otimes^{\mathbf{L}}K=0$, and this implies that the cone of $\alpha$ also vanishes.

Composing the inverse of $\alpha$ with the map

[TABLE]

we obtain a perfect obstruction theory

[TABLE]

Proposition 2.9**.**

Let $U\subset Y$ be an open subscheme such that $\omega_{U}$ is trivial. Then the map $\phi_{U}\colon\mathbb{E}_{U}\to\mathbb{L}_{\mathrm{Q}_{U}}$ is a symmetric perfect obstruction theory.

Proof.

Any choice of trivialisation $\omega_{U}\,\widetilde{\to}\,\mathscr{O}_{U}$ induces, by pullback along $U\times\mathrm{Q}_{U}\to U$, a trivialisation $\omega_{\pi}\,\widetilde{\to}\,\mathscr{O}_{U\times\mathrm{Q}_{U}}$, that we use to construct an isomorphism

[TABLE]

From now on the proof is similar to that of Theorem 2.5, except that we cannot use Grothendieck duality for $\pi$, since it is not proper. Thus we include full details.

Dualising and shifting the last displayed isomorphisms, we obtain

[TABLE]

We need to show that $(\mathbf{R}\pi_{\ast}\mathbb{H}_{U})^{\vee}[-1]=\mathbb{E}_{U}$. Note that, again by the projection formula along $a$, one has

[TABLE]

and moreover both complexes $\mathbb{H}_{Y}$ and $\mathbf{R}a_{\ast}\mathscr{O}_{U\times\mathrm{Q}_{U}}$ are canonically self-dual. Then

[TABLE]

The symmetry property $\theta_{U}^{\vee}[1]=\theta_{U}$ again follows from [5, Lemma 1.23]. ∎

Example 2.10**.**

Taking $Y=\mathbb{P}^{3}$, $U=\mathbb{A}^{3}$, $F$ an exceptional bundle on $\mathbb{P}^{3}$ of rank $r$, we see that $\operatorname{Quot}_{\mathbb{A}^{3}}(\mathscr{O}^{\oplus r},n)$ carries a symmetric perfect obstruction theory. As far as we know, it might not be possible to construct exceptional bundles on $\mathbb{P}^{3}$ of any given rank. However, $\operatorname{Quot}_{\mathbb{A}^{3}}(\mathscr{O}^{\oplus r},n)$ does have a symmetric obstruction theory for every $r$. This follows directly from its description as a critical locus [1, Thm. 2.6], that we recall in Section 4.1.

Aside 2.11**.**

The problems of constructing exceptional bundles and proving their stability are classical in Algebraic Geometry. By the foundational work of Drézet and Le Potier, all exceptional bundles on $\mathbb{P}^{2}$ are stable [12]. By work of Zube [33], the same is true for any K3 surface with Picard group $\mathbb{Z}$. This fact is used in loc. cit. to prove that any exceptional bundle on $\mathbb{P}^{3}$ is stable. Miró-Roig and Soares [24] prove that if $Y\subset\mathbb{P}^{n}$ is a smooth complete intersection $3$-fold of type $(d_{1},\ldots,d_{n-3})$, with $d_{1}+\cdots+d_{n-3}\leq n$ and $n\geq 4$, then any exceptional bundle on $Y$ is stable.

2.5. The stable case

Let $H$ be a polarisation on the $3$-fold $Y$, i.e. an ample class in $H^{2}(Y,\mathbb{Z})$. Assume $F$ is a $\mu_{H}$-stable (and rigid) vector bundle. Then the open immersion $\Psi_{n}$ of Proposition 2.8 factors through an open immersion

[TABLE]

where the target is the moduli space of $\mu_{H}$-stable sheaves with Chern character $v_{n}=\operatorname{ch}(F)-(0,0,0,n)$.

Remark 2.12**.**

The open immersion $\Phi_{n}$ is also closed. Indeed, $\mathcal{M}_{H}^{\mathrm{st}}(v_{n})$ is a separated scheme, so by properness of the Quot scheme $\Phi_{n}$ is a proper morphism. But a proper open immersion is a closed immersion. Hence $\Phi_{n}$ is the inclusion of a union of connected components.

Remark 2.13**.**

The perfect obstruction theory on the moduli space $\mathcal{M}_{H}^{\mathrm{st}}(v_{n})$ constructed by Thomas [32, Cor. 3.39] (in the case when there are no strictly $\mu_{H}$-semistable sheaves and $Y$ has an anticanonical section) pulls back via $\Phi_{n}$ to the one constructed in Theorem 2.5. For instance, in the Calabi–Yau case, the condition

[TABLE]

implies that there are no strictly semistable sheaves. In fact, it implies the stronger statement that there exists a universal sheaf over $\mathcal{M}^{\mathrm{st}}_{H}(v_{n})$, see [1, Cor. B.2] for a proof.

Example 2.14**.**

If $F=\mathscr{O}_{Y}$, one has $\operatorname{Quot}_{Y}(\mathscr{O}_{Y},n)=\operatorname{Hilb}^{n}Y$ and $\mathcal{M}^{\mathrm{st}}(1,0,0,-n)$, independent of the polarisation, is the moduli space of ideal sheaves (we are using that the determinant is fixed, thanks to $H^{1}(Y,\mathscr{O}_{Y})=0$). In this case the open immersion $\Phi_{n}$ of (2.11) is also surjective: this recovers the classical identification of $\operatorname{Hilb}^{n}Y$ with the moduli space of torsion free sheaves of Chern character $(1,0,0,-n)$.

3. Higher rank Donaldson–Thomas invariants

3.1. Calabi–Yau 3-folds

Let us recall from [1] the following weighted Euler characteristic calculation.

Theorem 3.1** ([1, Thm. A]).**

Let $Y$ be a smooth quasi-projective $3$-fold, $F$ a locally free sheaf of rank $r$. Then

[TABLE]

where $\mathsf{M}(q)=\prod_{m\geq 1}(1-q^{m})^{-m}$ is the MacMahon function.

Let now $Y$ be a projective Calabi–Yau $3$-fold, $F$ a simple rigid vector bundle. Set

[TABLE]

where $\mathsf{DT}_{F,n}$ is the degree of the virtual class constructed in Corollary 2.6, see (2.7).

Theorem 3.1 has the following immediate consequence.

Corollary 3.2**.**

If $F$ is a simple rigid vector bundle on a Calabi–Yau $3$-fold $Y$, then

[TABLE]

Proof.

Since the Quot scheme is proper and the obstruction theory constructed in Theorem 2.5 is symmetric, by Behrend’s theorem we have

[TABLE]

The result then follows from Theorem 3.1. ∎

3.2. The stable case

Classical Donaldson–Thomas theory is defined for the moduli space of stable sheaves $\mathcal{M}^{\mathrm{st}}_{H}(\alpha)$, where $\alpha\in H^{\ast}(Y,\mathbb{Q})$ is a given Chern character. If $F$ is a $\mu_{H}$-stable rigid vector bundle, (3.2) computes the virtual enumerative contribution of the connected component (cf. Remark 2.12)

[TABLE]

Therefore Equation (3.1) can be seen as an explicit example of (classical) higher rank DT invariants.

Example 3.3**.**

Recall that a vector bundle $F$ on a hypersurface $Y\subset\mathbb{P}^{r+1}$ is arithmetically Cohen–Macaulay if $H^{i}(Y,F(k))=0$ for $0<i<r$ and for all $k\in\mathbb{Z}$. By a result of Chiantini and Madonna [8, Thm. 1.3], every stable arithmetically Cohen–Macaulay rank $2$ bundle on a general quintic $Y\subset\mathbb{P}^{4}$ is rigid. Therefore, since $\chi(Y)=-200$, for any such $F$ Equation (3.1) yields

[TABLE]

This discussion motivates the following:

Problem 3.4.

Construct examples of stable rigid vector bundles on Calabi–Yau $3$-folds.

3.3. General $3$-folds

Let $Y$ be a smooth projective $3$-fold, $F$ a vector bundle of rank $r$. The numbers $\mathsf{DT}_{F,n}$ and their generating function $\mathsf{DT}_{F}(q)$ can be defined as in (2.7) whenever the virtual class is defined. In the rank $1$ case, one has

[TABLE]

See [23] for a proof in the toric case and [21, 22] for a general proof. We propose the following conjecture.

Conjecture 3.5**.**

Let $Y$ be a smooth projective $3$-fold, $F$ a vector bundle of rank $r$ on $Y$ such that the series $\mathsf{DT}_{F}(q)$ is defined. Then there is an identity

[TABLE]

Conjecture 3.5 does not seem to trivially follow from the existing arguments in the rank $1$ case. Besides the rank $1$ case, the formula is true in the Calabi–Yau case, by (3.1). We hope to get back to this question in the future.111Update: Conjecture 3.5 has recently been proven for a toric $3$-fold and an equivariant exceptional locally free sheaf $F$ by Fasola, Monavari and the author in [14, Thm. C]. The general case remains open.

4. The virtual motive of the Quot scheme

Throughout this section, we drop all assumptions on $(Y,F)$ we had previously. We let $Y$ be an arbitrary smooth quasi-projective $3$-fold, $F$ a vector bundle of rank $r$, and we consider $\operatorname{Quot}_{Y}(F,n)$. In this section we construct a virtual motive for this Quot scheme, i.e. a motivic weight

[TABLE]

such that applying the map $\chi$ of (1.2) yields

[TABLE]

where the second equality is equivalent to Theorem 3.1.

4.1. The local model

In this subsection we work on the local Calabi–Yau $3$-fold $Y=\mathbb{A}^{3}$. Fix $r\geq 1$ and $n\geq 0$. In [1, Thm. 2.6], it was proved that

[TABLE]

is a critical locus. In the case $r=1$ (corresponding to the Hilbert scheme of points) this was already known [3, Prop. 3.1]. In particular, $\operatorname{Quot}_{\mathbb{A}^{3}}(\mathscr{O}^{\oplus r},n)$ carries both the structures (symmetric obstruction theory, virtual motive) recalled in Section 1.

4.1.1. The critical structure on the local Quot scheme

We briefly review from [1] the critical structure on $\operatorname{Quot}_{\mathbb{A}^{3}}(\mathscr{O}^{\oplus r},n)$. The affine space

[TABLE]

parameterising triples of $n$ by $n$ matrices and $r$-tuples of $n$-vectors, has dimension $3n^{2}+rn$. It can be seen as the space of $(n,1)$-dimensional representations of the $3$-loop quiver endowed with $r$ framings issuing from an additional vertex $\infty$, cf. Figure 1.

The group $\operatorname{GL}_{n}$ acts freely on the open subscheme

[TABLE]

parameterising tuples $(A,B,C,v_{1},\ldots,v_{r})$ such that the $\mathbb{C}$-linear span of the vectors of the form

[TABLE]

has maximal dimension, i.e. it equals $\mathbb{C}^{n}$. It was proved in [1, Prop. 2.4] that $U_{r,n}$ can be identified with a subspace of stable framed representations of the $3$-loop quiver. The quotient

[TABLE]

is called non-commutative Quot scheme in [1], by analogy with the case $r=1$, giving rise to the non-commutative Hilbert scheme. It is a smooth quasi-projective variety of dimension $2n^{2}+rn$. Consider the function

[TABLE]

Theorem 4.1** ([1, Thm. 2.6]).**

There is a scheme-theoretic isomorphism

[TABLE]

Example 4.2**.**

The potential $f_{r,1}$ vanishes for any $r$, so $\operatorname{Quot}_{\mathbb{A}^{3}}(\mathscr{O}^{\oplus r},1)=\operatorname{Quot}^{1}_{r}$ is smooth of dimension $r+2$. On the other hand, unlike the Hilbert scheme $\operatorname{Hilb}^{n}\mathbb{A}^{3}$, which is nonsingular for $n\leq 3$, the Quot scheme $\operatorname{Quot}_{\mathbb{A}^{3}}(\mathscr{O}^{\oplus r},r)$ is singular for all $r>1$. Indeed, the submodule $S=(x,y,z)^{\oplus r}\subset\mathbb{C}[x,y,z]^{\oplus r}$ defines a point whose tangent space has dimension $3r^{2}$. But $3r^{2}=\dim\operatorname{Quot}^{r}_{r}>\dim\operatorname{Quot}_{\mathbb{A}^{3}}(\mathscr{O}^{\oplus r},r)$ since $f_{r,r}\neq 0$. Even in rank $1$, if we replace $\mathscr{O}_{\mathbb{A}^{3}}$ by the ideal sheaf of a line $L\subset\mathbb{A}^{3}$, the Quot scheme $\operatorname{Quot}_{\mathbb{A}^{3}}(\mathscr{I}_{L},2)$ turns out to be singular, cf. [10, Example 2.7].

The virtual motive induced by the critical structure (4.2) takes the form

[TABLE]

and we shall see (cf. Lemma 4.4) that it lives in the monodromy-free subring $\mathcal{M}_{\mathbb{C}}\subset\mathcal{M}^{\hat{\mu}}_{\mathbb{C}}$. Let us form the generating function

[TABLE]

The following computation was carried out following step by step the rank $1$ calculation by Behrend–Bryan–Szendrői [3].

Proposition 4.3** ([26, Prop. 2.3.6]).**

There is an identity

[TABLE]

A new proof of Proposition 4.3, using wall-crossing for framed quiver representations, appeared recently in [7]. We next compute (cf. Corollary 4.9) the motive (4.3) and we show it is determined, via the power structure (cf. Section 1.2.3), by the virtual motivic contributions of the “punctual strata”, just as in the rank $1$ case — see [3, Section 3] and [10, Section 3]. This will allow us to define a virtual motive for all pairs $(Y,F)$ where $Y$ is a smooth quasi-projective $3$-fold and $F$ is a rank $r$ vector bundle on $Y$.

4.1.2. The virtual motive of the Quot scheme of $\mathbb{A}^{3}$

Let us fix $r\geq 1$ and for convenience let us shorten $\mathrm{Q}_{r,n}=\operatorname{Quot}_{\mathbb{A}^{3}}(\mathscr{O}^{\oplus r},n)$. Consider the Quot-to-Chow morphism

[TABLE]

Lemma 4.4**.**

The absolute motivic vanishing cycle $\phi_{f_{r,n}}$ satisfies the relation

[TABLE]

and the direct image along $\sigma_{n}$ of the motivic vanishing cycle is monodromy-free,

[TABLE]

Proof.

Let $\mathbb{T}=\mathbb{G}_{m}^{3}$ be the $3$-dimensional torus. The function $f_{r,n}$ is equivariant with respect to the primitive character $\chi(t)=t_{1}t_{2}t_{3}$, and a standard argument [3] shows that the action of the diagonal subgroup $\mathbb{G}_{m}\subset\mathbb{T}$ is circle compact. Therefore the formula for $[\phi_{f_{r,n}}]$ follows from [3, Thm. B1].

Let $L_{3}$ be the 3-loop quiver, i.e. the quiver obtained from the one in Figure 1 by removing all framings $\infty\to 0$. The function $(A,B,C)\mapsto\operatorname{Tr}A[B,C]$ on the space $\operatorname{Rep}_{n}(L_{3})$ of $n$-dimensional representations of $L_{3}$ is reduced. This implies that $f_{r,n}^{-1}(0)\subset\operatorname{Quot}^{n}_{r}$ is a reduced hypersurface. Let $a\colon\mathrm{Q}_{r,n}\to Z$ be the affinisation of the Quot scheme. Then, again by [3, Thm. B1], the direct image $a_{\,!\,}[\phi_{f_{r,n}}]_{\mathrm{Q}_{r,n}}$ is monodromy-free. Since $\operatorname{Sym}^{n}\mathbb{A}^{3}$ is affine, $\sigma_{n}$ factors through $a$, thus $\sigma_{n\,!\,}[\phi_{f_{r,n}}]_{\mathrm{Q}_{r,n}}$ is also monodromy-free. ∎

The punctual Quot scheme $\operatorname{Quot}_{\mathbb{A}^{3}}(\mathscr{O}^{\oplus r},n)_{0}\subset\mathrm{Q}_{r,n}$ is the locus of quotients $\mathscr{O}_{\mathbb{A}^{3}}^{r}\twoheadrightarrow Q$ such that $Q$ is entirely supported at the origin $0\in\mathbb{A}^{3}$. It is the fibre of $\sigma_{n}$ over the point $n\cdot 0\in\operatorname{Sym}^{n}_{(n)}\mathbb{A}^{3}$. We use the special notation

[TABLE]

for its virtual motivic contribution (see Notation 1.9 for the definition of the right hand side), and we form the generating function

[TABLE]

Define motivic weights

[TABLE]

by the identity

[TABLE]

Theorem 4.5**.**

There is an identity

[TABLE]

Proof.

The same analysis of [10, Sec. 3] shows that the relative virtual motives of $\mathrm{Q}_{r,n}$, viewed as relative classes over $\operatorname{Sym}(\mathbb{A}^{3})$, are generated under $\operatorname{Exp}_{\cup}$ by the classes $\Omega_{r,n}$ defined in (4.5), extended by the small diagonal. In other words, if $\Delta_{n}\colon\mathbb{A}^{3}\to\operatorname{Sym}^{n}\mathbb{A}^{3}$ denotes the small diagonal, a stratification argument combined with [10, Prop. 1.12] yields an identity

[TABLE]

in $\mathcal{M}_{\operatorname{Sym}(\mathbb{A}^{3})}$.

Consider the map $\#\colon\operatorname{Sym}(\mathbb{A}^{3})\to\mathbb{N}$ sending $\operatorname{Sym}^{n}\mathbb{A}^{3}$ to the point $n$. Its direct image $\#_{!}$ is described in (1.7). By applying $\#_{!}$ to both sides of (4.6), and using [10, Prop. 1.12] along with Equation (4.5), we deduce the identity

[TABLE]

Next, we prove that $\Omega_{r,n}$ is effective for all $r$, $n$. A straightforward calculation along the lines of [3, Thm. 4.3] allows us to verify, starting from (4.4), that

[TABLE]

By Equation (4.7), this gives

[TABLE]

Combining (4.5) with the injectivity of ${}^{\diamond}\!\!\operatorname{Exp}$ (see [10, Lemma 1.11]), an elementary comparison shows that

[TABLE]

which belongs to $\mathcal{M}_{\mathbb{C}}^{\operatorname{eff}}\subset\mathcal{M}_{\mathbb{C}}$ for every $r$ and $n$. It follows that the classes $(-1)^{rn}\mathsf{P}_{r,n}$ are effective (because the plethystic exponential preserves effectiveness). This implies that

[TABLE]

so the result follows from Equation (4.7). ∎

Remark 4.6**.**

Equation (4.6) is the analogue of [10, Thm. 3.17]. The argument needed here is actually easier (and more similar to the setup of [3]) than the one in [10]. Indeed, in the present situation, there is only one punctual contribution, whereas in [10] two types of punctual contributions had to be considered.

Remark 4.7**.**

For $r=1$ we recover the effective classes

[TABLE]

determining, via the identity \operatorname{Exp}\bigl{(}\sum_{n\geq 1}\Omega_{1,n}\cdot t^{n}\bigr{)}=\sum_{n\geq 0}\,[\operatorname{Hilb}^{n}(\mathbb{A}^{3})_{0}]_{\operatorname{vir}}(-t)^{n}, the virtual motives of the punctual Hilbert scheme of $\mathbb{A}^{3}$ defined in [3].

Remark 4.8**.**

Since the classes in Equation (4.9) are effective, the ‘$\diamond$’ decoration in Equation (4.5) can be removed, and we obtain the identity

[TABLE]

This relation can be viewed as the local motivic analogue of the enumerative identity

[TABLE]

where, for a projective $3$-fold $Y$, the number $\mathrm{N}^{Y}_{n,0}\in\mathbb{Q}$ is the virtual count of semistable sheaves $E\in\operatorname{Coh}(Y)$ supported in dimension [math] and with $\chi(E)=n$.

Corollary 4.9**.**

For all $r\geq 1$ and $n\geq 0$, there is an identity

[TABLE]

Proof.

By the proof of Theorem 4.5, the motives $(-1)^{ri}\mathsf{P}_{r,i}$ are effective. The result follows directly from the theorem and the power structure formula for an effective power series, cf. (1.6). ∎

4.2. Virtual motives for arbitrary $3$-folds

Let $Y$ be a smooth quasi-projective $3$-fold, $F$ a vector bundle of rank $r$.

Definition 4.10**.**

We define the motivic weights $[\operatorname{Quot}_{Y}(F,n)]_{\operatorname{vir}}\in\mathcal{M}_{\mathbb{C}}$ by the identity

[TABLE]

Note that for $Y=\mathbb{A}^{3}$ this definition reconstruct the virtual motive of $\operatorname{Quot}_{\mathbb{A}^{3}}(\mathscr{O}^{\oplus r},n)$ by Theorem 4.5.

Let us form the motivic partition function

[TABLE]

Lemma 4.11**.**

The motivic weight $[\operatorname{Quot}_{Y}(F,n)]_{\operatorname{vir}}$ is a virtual motive for $\operatorname{Quot}_{Y}(F,n)$.

Proof.

The chain of equalities

[TABLE]

implies that

[TABLE]

The claim then follows by substituting $t\to(-1)^{r}t$ and comparing with Theorem 3.1. ∎

We now derive a formula for $\mathsf{Z}_{r}(Y,(-1)^{r}t)$ in terms of the motivic exponential.

Theorem 4.12**.**

Let $Y$ be a smooth $3$-fold, $F$ a vector bundle of rank $r$. Then

[TABLE]

Proof.

We have

[TABLE]

where we used Formula (4.8) in the second equality. Recall from Example 1.10 that $[Y]_{\operatorname{vir}}=\mathbb{L}^{-3/2}[Y]$. Then the formula follows by $[Y]_{\operatorname{vir}}\cdot[\mathbb{P}^{r-1}]_{\operatorname{vir}}=[Y\times\mathbb{P}^{r-1}]_{\operatorname{vir}}$. ∎

Remark 4.13**.**

The formula of Theorem 4.12 can also be rewritten as

[TABLE]

Remark 4.14**.**

An abstract power structure formula for the naive motive of $\operatorname{Quot}_{Y}(F,n)$ was given in [29], generalising the work of Gusein-Zade, Luengo and Melle-Hernández for the case of $\operatorname{Hilb}^{n}Y$ [17].

4.3. Motivic Donaldson–Thomas invariants

Let $F$ be a simple rigid vector bundle on a Calabi–Yau $3$-fold $Y$. Then the motivic weight $[\operatorname{Quot}_{Y}(F,n)]_{\operatorname{vir}}$ of Definition 4.10 can be seen as a (rank $r$) motivic Donaldson–Thomas invariant, for it refines the enumerative invariant $\mathsf{DT}_{F,n}$ computed by (3.2). Similarly, the motivic partition function $\mathsf{Z}_{r}(Y,t)$ of Theorem 4.12 can be seen as a motivic refinement of the enumerative generating function $\mathsf{DT}_{F}(q)$ computed in (3.1).

An explicit example of such higher rank refinement (in the context of stable sheaves) is provided by a rank $2$ arithmetically Cohen–Macaulay stable bundle $F$ on a generic quintic $3$-fold $Y$ in $\mathbb{P}^{4}$, cf. Example 3.3.

4.3.1. An open question

Let $(Y,H)$ be a polarised Calabi–Yau $3$-fold. The moduli space of stable sheaves $\mathcal{M}^{\mathrm{st}}_{H}(\alpha)$ with Chern character $\alpha$ has the structure of an oriented d-critical locus in the sense of [20, Def. 2.31]. See [25] for a proof of existence of orientations, i.e. square roots of the virtual canonical bundle. Let $F$ be a stable rigid vector bundle on $Y$. Then the connected component

[TABLE]

inherits an oriented d-critical structure. In particular, by the results of [6], each orientation $L$ induces a canonical virtual motive

[TABLE]

It is an interesting question to check whether there exists an orientation $L$ such that the induced virtual motives $\mathsf{MF}^{L}_{\operatorname{Quot}_{Y}(F,n)}$ agree with the ones defined in this paper (cf. Definition 4.10). As far as we know, this is still unknown in the rank $1$ case [3], i.e. for the Hilbert scheme of points.

4.4. Vanishing cycle cohomology

It follows from [9, Thm. 6.3] and the description of the space $\operatorname{Quot}_{\mathbb{A}^{3}}(\mathscr{O}^{\oplus r},n)$ as a fine moduli space of quiver representations [1, Prop. 2.4], that the mixed Hodge structure on the total compactly supported vanishing cycle cohomology

[TABLE]

is pure of Tate type for all $n$. Here $f_{r,n}\colon\operatorname{Quot}^{n}_{r}\to\mathbb{A}^{1}$ is the regular function defined in (4.1) and

[TABLE]

is the perverse sheaf of vanishing cycles. Just as in [9, Example 6.4], thanks to purity we can evaluate the Hodge polynomial of the Quot scheme starting from the identity

[TABLE]

obtained by applying the renormalisation $t\mapsto\mathbb{L}^{-r/2}t$ to Equation (4.4). According to [9, Section 1.1], the Hodge polynomial (more precisely, the Hodge series) of a cohomologically graded mixed Hodge structure $\mathscr{L}$ is the formal power series

[TABLE]

The E-series is given by $\mathsf{E}(\mathscr{L};x,y)=\mathsf{h}(\mathscr{L};x,y,-1)$, and the weight series is defined by the further specialisation

[TABLE]

where $q$ keeps track of cohomological degree. We have

[TABLE]

after the substitution $q^{2}=xyz^{2}$. Thus, by specialising $\mathbb{L}\to q^{2}$, we deduce from (4.10) the identity

[TABLE]