Subelliptic boundary value problems and The $G$-Fredholm Property

TL;DR

This paper studies the $G$-Fredholm properties of the Laplacian on complex manifolds with boundary, showing finite $G$-dimension kernels and finite codimension images under group actions, extending classical elliptic theory to subelliptic settings.

Contribution

It establishes that the Laplacian on certain complex manifolds with boundary has the $G$-Fredholm property, generalizing elliptic operator results to subelliptic and group-invariant contexts.

Findings

The kernel of the Laplacian has finite $G$-dimension.

The image of the Laplacian contains a closed, finite codimension, $G$-invariant subspace.

Similar properties hold for the boundary Laplacian $ox_b$.

Abstract

Let $G$ be a unimodular Lie group, $X$ a compact manifold with boundary, and $M$ be the total space of a principal bundle $G\to M\to X$ so that $M$ is also a complex manifold satisfying a local subelliptic estimate. In this work, we show that if $G$ acts by holomorphic transformations in $M$, then the Laplacian $\square=\bar\partial^{*}\bar\partial+\bar\partial\bar\partial^{*}$ on $M$ has the following properties: The kernel of $\square$ restricted to the forms $\Lambda^{p,q}$ with $q>0$ is a closed, $G$-invariant subspace in $L^{2}(M,\Lambda^{p,q})$ of finite $G$-dimension. Secondly, we show that if $q>0$, then the image of $\square$ contains a closed, $G$-invariant subspace of finite codimension in $L^{2}(M,\Lambda^{p,q})$. These two properties taken together amount to saying that $\square$ is a $G$-Fredholm operator. In similar circumstances, the boundary Laplacian $\square_b$ has similar properties.

Equations

$${\dim }_G(L)={\mathrm{Tr}}_G(P_L).$$

$$(L_{f}u)(t)={\int }_{G}f(s)(L_{s}u)(t)ds={\int }_{G}f(s)u(s^{-1}t)ds.$$

$${\mathrm{tr}}_G(L_f^{\ast }{L}_f)={\int }_G|f(s){|}^{2}ds,$$

$$L^2(M)\cong L^2(G)\otimes L^2(X).$$

$$\mathcal{B}(L^2(M){)}^G\cong \mathcal{B}(L^2(G){)}^G\otimes \mathcal{B}(L^2(X))\cong {\mathcal{L}}_G\otimes \mathcal{B}(L^2(X)),$$

$$L^2(M)\cong L^2(G)\otimes L^2(X)\cong \underset{l\in \mathbb{N}}{⨁}{L}^2(G)\otimes {\psi }_l.$$

$$A\leftrightarrow [A_{lm}{]}_{lm}=[L_{h_{lm}}{]}_{lm}.$$

$${\mathrm{Tr}}_G(A)=\sum _{l\in \mathbb{N}}{\mathrm{tr}}_G(A_{ll}).$$

$${\dim }_{G}L={\mathrm{Tr}}_{G}P.$$

$$\varphi =\sum _{|I|=p,|J|=q}{\varphi }_{I,J}d{z}^I\wedge d{\bar{z}}^J$$

$$\bar{\partial }\varphi =\sum _{|I|=p,|J|=q}\sum _{k=1}^n\frac{\partial {\varphi }_{I,J}}{\partial {\bar{z}}^k}d{\bar{z}}^k\wedge d{z}^I\wedge d{\bar{z}}^J$$

$$0⟶{\Lambda }^{p,0}\stackrel{{\bar{\partial }}_{p,0}}{⟶}{\Lambda }^{p,1}\stackrel{{\bar{\partial }}_{p,1}}{⟶}\dots \stackrel{{\bar{\partial }}_{p,q-1}}{⟶}{\Lambda }^{p,q}\stackrel{{\bar{\partial }}_{p,q}}{⟶}\dots \stackrel{{\bar{\partial }}_{p,n-1}}{⟶}{\Lambda }^{p,n}\stackrel{{\bar{\partial }}_{p,n}}{⟶}0,$$

$$L^2{\bar{H}}^{p,q}(M)=\ker ({\bar{\partial }}_{p,q})/\overline{\mathrm{im}({\bar{\partial }}_{p,q-1})}.$$

$$Q(\varphi ,\psi )=⟨\bar{\partial }\varphi ,\bar{\partial }\psi ⟩+⟨{\bar{\partial }}^{\ast }\varphi ,{\bar{\partial }}^{\ast }\psi ⟩+⟨\varphi ,\psi ⟩.$$

$$\square ={\square }_{p,q}={\bar{\partial }}^{\ast }\bar{\partial }+\bar{\partial }{\bar{\partial }}^{\ast }\mathrm{on}\mathrm{Dom}\square \subset L^2(M,{\Lambda }^{p,q}).$$

$$\mathrm{Dom}(F_{p,q})=\overline{\{\varphi \in {\mathcal{D}}^{p,q}\mid \bar{\partial }\varphi \in {\mathcal{D}}^{p,q+1}\}}$$

$$L^2(M,{\Lambda }^{\bullet })=\overline{\mathrm{im}\bar{\partial }}\oplus \ker \square \oplus \overline{\mathrm{im}{\bar{\partial }}^{\ast }}\ker \bar{\partial }=\overline{\mathrm{im}\bar{\partial }}\oplus \ker \square .$$

$$L^2{\bar{H}}^{p,q}(M)=\ker {\square }_{p,q}$$

$${\dim }_G{L}^2{\bar{H}}^{p,q}(M)={\dim }_G\ker {\square }_{p,q}<\infty .$$

$$\Vert \varphi {\Vert }_{H^{-s}(M)}=\sup \left\{\frac{|⟨\varphi ,\psi ⟩|}{\Vert \psi {\Vert }_{H^s(M)}}\mid \psi \in C_c^{\infty }(M)\right\}(s>0).$$

$$\tilde{u}(\tau ,\rho )=\frac{1}{(2\pi {)}^{(2n-1)/2}}{\int }_{{\mathbb{R}}^{2n-1}}dt{e}^{-i⟨t,\tau ⟩}u(t,\rho ),$$

$${\Lambda }_{\mathbf{t}}^{s}u(t,\rho )=\frac{1}{(2\pi {)}^{(2n-1)/2}}{\int }_{{\mathbb{R}}^{2n-1}}d\tau e^{i⟨t,\tau ⟩}(1+|\tau {|}^2{)}^{s/2}\tilde{u}(\tau ,\rho )$$

$$|||u||{|}_s^2={\int }_{{\mathbb{R}}^{2n-1}}d\tau {\int }_{-\infty }^{0}d\rho (1+|\tau {|}^2{)}^s|\tilde{u}(\tau ,\rho ){|}^2.$$

$$|||Du||{|}_s^2=\sum _1^{2n}|||D^{j}u||{|}_s^2+|||u||{|}_s^2\approx |||u||{|}_{s+1}^2+|||D_{\rho }u||{|}_s^2.$$

$$\Vert \varphi {\Vert }_{ϵ}^2\le C(\Vert \bar{\partial }\varphi {\Vert }^2+\Vert {\bar{\partial }}^{\ast }\varphi {\Vert }^2+\Vert \varphi {\Vert }^2),$$

$$G⟶M⟶X$$

$$|||\varphi ||{|}_{ϵ}^2\le C^{\prime }(\Vert \bar{\partial }\varphi {\Vert }^2+\Vert {\bar{\partial }}^{\ast }\varphi {\Vert }^2+\Vert \varphi {\Vert }^2)(\varphi \in {\mathcal{D}}_{V^{\prime }}^{p,q}).$$

$$(L_1,L_2)\mapsto ⟨\partial \bar{\partial }\rho ,L_1\wedge {\bar{L}}_2⟩.$$

$$E_U(\varphi {)}^2=\sum _{IJk}\Vert {\bar{L}}_k{\varphi }_{IJ}{\Vert }^2+{\int }_{bM}|\varphi {|}^2+\Vert \varphi {\Vert }^2.$$

$$|||D\varphi ||{|}_{-1/2}^2\lesssim E(\varphi {)}^2.$$

Full text

Supported by FWF grant P19667, Mapping Problems in Several Complex Variables

MR Classification numbers: 32W05, 35H20, 43A30

Keywords: $\bar{\partial}$-Neumann Problem, Subelliptic operators

1. Introduction

Let $\mathcal{H}_{1}$ and $\mathcal{H}_{2}$ be Hilbert spaces and let $\mathcal{B}(\mathcal{H}_{1},\mathcal{H}_{2})$ be the space of bounded linear operators $A:\mathcal{H}_{1}\to\mathcal{H}_{2}$. An operator $A\in\mathcal{B}(\mathcal{H}_{1},\mathcal{H}_{2})$ is said to be Fredholm if first, the kernel of $A$ is finite-dimensional, and second the image of $A$ is closed and has finite codimension. An application of the open mapping theorem shows that the closedness requirement on the image is redundant. A well-known example of Fredholm operators due to Riesz: if $C$ is a compact operator then ${\bf 1}-C$ is Fredholm. It is easy to see that the Fredholm property is equivalent to invertiblility modulo finite-rank operators or compact operators.

The main example of such a situation is given by elliptic differential operators acting in sections of vector bundles over compact manifolds. Choosing $\mathcal{H}_{1}$ and $\mathcal{H}_{2}$ to be appropriate Sobolev spaces of these sections, we find that elliptic operators are Fredholm.

If $M$ is a noncompact manifold (possibly with boundary) and $A$ an elliptic differential operator on $M$, then it is not necessarily the case that $A$ be Fredholm. That is, the kernel and/or cokernel of $A$ may be infinite-dimensional and/or the image of $A$ may not be closed. In particular, the index as defined above may not be well-defined, but there are notions generalizing the Fredholm property and the index. In this paper we will extend the results in [P1] using one of these generalized Fredholm properties that makes sense when there is a free action of a unimodular Lie group $G$ on $M$. Making appropriate choices of metric on $M$ and in the vector bundles over $M$ and using a Haar measure on $G$, we obtain Hilbert spaces of sections on which the $G$-action is unitary. This action allows us to define a trace ${\rm Tr}_{G}$ in the algebra of operators commuting with the action of $G$. Restricting this trace to orthogonal projections $P_{L}$ onto $G$-invariant subspaces $L$ provides a dimension function $\dim_{G}$ given by

[TABLE]

Generalizing the previous definition of the Fredholm property, a $G$-invariant operator $A:\mathcal{H}_{1}\to\mathcal{H}_{2}$ is said to be $G$-Fredholm if $\dim_{G}\ker A<\infty$ and if there exists a closed, invariant subspace $Q\subset{\rm im}(A)$ so that $\dim_{G}(\mathcal{H}_{2}\ominus Q)<\infty$.

In [P1] it was established that if $M$ is a strongly pseudoconvex complex manifold $M$ with a unimodular Lie group $G$ acting on it freely, by holomorphic transformations so that $M/G$ is compact, then, for $q>0$, the Kohn Laplacian $\square$ is $G$-Fredholm in $L^{2}(M,\Lambda^{p,q})$.

Here we will relax the requirement on the boundary, requiring only that each point of the boundary have a neighborhood in which a subelliptic estimate holds, as defined below. In addition, we will in detail demonstrate a similar result for the boundary Laplacian $\square_{b}$. The main results of this paper are.

Theorem 1.1**.**

Let $M$ be a complex manifold with boundary satisfying a local subelliptic property. Let $G$ be a unimodular Lie group acting freely by holomorphic transformations on $M$ so that $M/G$ is compact. Then, for $q>0$, the Kohn Laplacian $\square$ in $L^{2}(M,\Lambda^{p,q})$ and the boundary Laplacian $\square_{b}$ in $L^{2}(bM,\Lambda^{p,q})$ are $G$-Fredholm.

Remark 1.2*.*

Natural examples of manifolds satisfying the hypotheses of the theorem are the $G$-complexifications of group actions on manifolds as constructed in [HHK]. The unimodularity of $G$ is necessary for the definition of the $G$-Fredholm property.

The $\bar{\partial}$-Neumann problem was proposed by Spencer in the 1950s as a method of obtaining existence theorems for holomorphic functions. Morrey in [Mo] introduced the key basic estimate and the problem was solved by Kohn in [K1]. We use and develop variants of the techniques in [KN, FK, E] in this work as well as review other approaches to the problem.

The generalized Fredholm property which we use was first introduced in an abstract setting by M. Breuer [B]. In [GHS], it is shown that if $\Gamma$ is taken to be a discrete group and $M$ strongly pseudoconvex, then the Kohn Laplacian $\square$ is $\Gamma$-Fredholm. Note that the natural boundary value problem for $\square$ (called the $\bar{\partial}$-Neumann problem) is not elliptic, but only subelliptic. In [P1] we extended this result from [GHS] to the situation in which $G$ is a unimodular Lie group, with the manifold still strongly pseudoconvex.

Recent works on covering spaces extending results from [GHS] can be found in [Br1, Br2, Br3, Br4], some of which are related to the Shafarevich conjecture (which asserts that that the universal covering of a smooth projective variety $X$ is holomorphically convex). More geometric and analytic are [TCM1] dealing with the case in which $M$ is only assumed weakly pseudoconvex. Using cohomological techniques and holomorphic Morse inequalities, the same authors obtain a lower bound for the $\Gamma$-dimension of the space of $L^{2}$ sections and upper bounds for the $\Gamma$-dimensions of the higher cohomology groups. In [TCM2], it is shown that the von Neumann dimension of the space of $L^{2}$ holomorphic sections is bounded below under curvature conditions on $M$.

This work is also connected to recent results in another direction. By Riesz, the $G$-Fredholm property of $\square$ is a natural analogue of the compactness property of the Neumann operator $N$ (the inverse of the restriction of $\square$ to $(\ker\square)^{\perp}$) in the case that the manifold in question is noncompact. The compactness property of $N$ has been studied extensively in settings in which $M$ is not strongly pseudoconvex. This work is a first step toward obtaining similar results on $G$-manifolds.

In a series of papers, Catlin, Crist, Fu, McNeal, Straube and others (see the excellent reviews [DK, FS]) have given sufficient conditions for, and obstacles to, compactness of $N$, the Neumann operator, assuming that $M$ is compact. In particular, in this work it is remarked compactness is a local property. Roughly speaking, the $\bar{\partial}$-Neumann operator $N_{q}$ on $\Omega$ is compact if and only if every boundary point has a neighborhood $U$ such that the corresponding $\bar{\partial}$-Neumann operator on $U\cap\Omega$ is compact. It would be interesting to see if or in what sense this notion generalizes.

Also, in a more speculative vein, can results analogous to those in [CF] be obtained for the noncompact case by the methods of [SML]?

In section 2 we will introduce the $G$-trace for invariant operators in Hilbert $G$-modules. Section 3 contains a description of abstract $G$-Fredholm operators and several useful properties. Section 4 treats the relevant results from the theory of the $\bar{\partial}$-Neumann problem. In section 5 we prove that $\square$ is $G$-Fredholm and deduce the finite-dimensionality of the reduced Dolbeault cohomology for $q>0$. We also explore some easy consequences of the main theorem regarding the operator $\bar{\partial}$ on functions. In the appendix there is a derivation of a local a priori estimate for the Laplacian.

2. Preliminaries

2.1. The $G$-Fredholm property

If in a Hilbert space $\mathcal{H}$ there is a strongly continuous action of a group $R_{s}:\mathcal{H}\to\mathcal{H}$, $(s\in G)$, we denote the space of $G$-equivariant bounded linear operators in $\mathcal{H}$ by $\mathcal{B}(\mathcal{H})^{G}$. In other words $P\in\mathcal{B}(\mathcal{H})^{G}$ if $P\in{\mathcal{B}}(\mathcal{H})$ and $R_{s}P=PR_{s}$ for every $s\in G$. Closed, invariant subspaces in a Hilbert space can be obtained as images of projections $P\in\mathcal{B}(L^{2}(M))^{G}$. Let us restrict our attention to a group acting on itself.

For any $s\in G$ define left and right translations $L_{s},R_{s}:L^{2}(G)\to L^{2}(G)$ by $(L_{s}u)(t)=u(s^{-1}t)$, $(R_{s}u)(t)=u(ts)$. For $f\in L^{1}(G)$ and $u\in L^{2}(G)$, let

[TABLE]

The set $\{L_{f}\mid f\in L^{1}(G)\}$ forms an associative algebra of bounded operators in $L^{2}(G)$ which are right-invariant (i.e. commute with right translations). The weak closure of this algebra, $\mathcal{L}_{G}\subset\mathcal{B}(L^{2}(G))$ is a von Neumann algebra by the bicommutant theorem. We will also need to consider operators $L_{f}$ for $f\in L^{2}(G)$. These are defined on $C^{\infty}_{c}(G)$ and we may try to extend them by continuity to $L^{2}(G)$. This is not always possible, but we will be concerned only with those $L_{f}$ which are bounded or can be extended to bounded linear operators in $L^{2}(G)$. It follows from the Schwartz kernel theorem that any bounded right-invariant operator in $L^{2}(G)$ can be presented in the form $L_{f}$ for a distribution $f$ on $G$.

We will need the following fact from about group von Neumann algebras (cf. [P], sections 5.1 and 7.2). There is a unique trace ${\rm tr}_{G}$ on $\mathcal{L}_{G}\subset\mathcal{B}(L^{2}(G))$ agreeing with

[TABLE]

whenever $L_{f}\in\mathcal{B}(L^{2}(G))$ and $f\in L^{2}(G).$ Furthermore, ${\rm tr}_{G}(A^{*}A)<\infty$ if and only if there is an $f\in L^{2}(G)$ for which $A=L_{f}\in\mathcal{B}(L^{2}(G))$. If we define $\tilde{f}(t)=\bar{f}(t^{-1})$, and if $f_{k},g_{k}\in L^{2}(G)$, $k=1,\dots,N$, then the operator $A=\sum_{1}^{N}L_{\tilde{f}_{k}}L_{g_{k}}$ is in ${\rm Dom}({\rm tr}_{G})$. Furthermore, $A$ takes the form $A=L_{h}$ with $h$ continuous and ${\rm tr}_{G}(L_{h})=h(e)$.

Note that the unimodularity of the group is necessary for the trace property of ${\rm tr}_{G}$.

Now let us consider free group actions on a manifold. Let $G$ be a Lie group and $G\to M\overset{p}{\to}X$ be a principal $G$-bundle with compact base $X$. In particular, this means that we have a free right action of $G$ on $M$ with quotient space $X$, and $p:M\to X$ is the canonical projection. Having a smooth free action of $G$ on a manifold $M$ with a $G$-invariant measure $d{\bf x}$, and fixing a Haar measure $dt$ on $G$, we obtain a natural quotient measure $dx$ on $X=M/G$ which allows us to present $L^{2}(M)$ in the form

[TABLE]

It follows that we have a decomposition of the von Neumann algebra of bounded invariant operators

[TABLE]

where we have made the identification $\mathcal{L}_{G}\cong\mathcal{B}(L^{2}(G))^{G}$. In order to measure the invariant subspaces of $L^{2}(M)$, we will use a trace on $\mathcal{L}_{G}\otimes\mathcal{B}(L^{2}(X))$. It is true that there exists a natural normal, faithful, semifinite trace on this algebra. It is denoted ${\rm Tr}_{G}$ and may be constructed as follows.

Let $(\psi_{l})_{l\in\mathbb{N}}$ be an orthonormal basis for $L^{2}(X)$. Then

[TABLE]

Denoting by $P_{m}$ the projection onto the $m^{th}$ summand, we obtain a matrix representation of $A\in\mathcal{B}(L^{2}(M))$ with elements $A_{lm}=P_{l}AP_{m}\in\mathcal{B}(L^{2}(G))$. If $A\in\mathcal{B}(L^{2}(M))^{G}$, then these matrix elements are invariant operators in $L^{2}(G)$, and so there exist distributions $h_{lm}$ on $G$ so that $A\in\mathcal{B}(L^{2}(M))^{G}$ has a matrix representation

[TABLE]

For positive $A\in\mathcal{B}(L^{2}(M))^{G}$ define

[TABLE]

The functional ${\rm Tr}_{G}$ is a normal, faithful, and semifinite trace and is independent of the basis $(\psi_{l})_{l}$ used in its construction, cf. Section V.2 of [T].

Using this trace, we can define the $G$-dimension of a closed, invariant subspace $L\subset L^{2}(M)$ as follows. For such a subspace let $P$ be the self-adjoint projection onto $L$. Then $P$ is $G$-invariant and so we may define

[TABLE]

The $G$-dimension has the usual properties of a dimension if it is defined. For example, the $G$-dimension respects the orthogonal sum. Also, if $L_{1},L_{2},L$ are closed, invariant subspaces of $L^{2}(M)$ such that ${\rm dim}_{G}L_{1}>\dim_{G}(L\ominus L_{2})$, then $L_{1}\cap L_{2}\neq\{0\}$ and ${\rm dim}_{G}L_{1}\cap L_{2}\geq{\rm dim}_{G}L_{1}-\dim_{G}(L\ominus L_{2})$. See [GHS], Lemma (2.1).

We give now our formal definition of the $G$-Fredholm property.

Definition 2.1**.**

Let $L_{0},\ L_{1}$ be Hilbert spaces on which a unimodular group $G$ acts strongly continuously by unitary transformations, and let $A:L_{0}\to L_{1}$ be a closed densely-defined linear operator commuting with the action of $G$. Such an operator is called $G$-Fredholm if the following conditions are satisfied:

• •

${\rm dim}_{G}\ker A<\infty$

• •

there exists a $G$-invariant closed subspace $Q\subset L_{1}$ so that $Q\subset{\rm im}\ A$ and ${\rm codim}_{G}\ Q={\rm dim}_{G}(L_{1}\ominus Q)<\infty.$

Consequences of this definition are collected in [P1, S].

2.2. The $\bar{\partial}$-Neumann problem

Let $M$ be a complex manifold with boundary. For any integers $p,q$ with $1\leq p,q\leq n$ denote by $C^{\infty}(M,\Lambda^{p,q})$ the space of all $C^{\infty}$ forms of type $(p,q)$ on $M$, i.e. the forms which can be written in local complex coordinates $(z_{1},z_{2},\dots,z_{n})$ as

[TABLE]

where $dz^{I}=dz^{i_{1}}\wedge\dots\wedge dz^{i_{p}}$, $dz^{J}=d\bar{z}^{j_{1}}\wedge\dots\wedge d\bar{z}^{j_{q}}$, $I=(i_{1},\dots,i_{p})$, $J=(j_{1},\dots,j_{q})$, $i_{1}<\dots<i_{p}$, $j_{1}<\dots<j_{q}$, and the $\phi_{I,J}$ are smooth functions in local coordinates 111These sums will be understood to be over increasing multiindices.. For such a form $\phi$, the value of $\bar{\partial}\phi$ is

[TABLE]

so $\bar{\partial}=\bar{\partial}|_{p,q}$ defines a linear map $\bar{\partial}:C^{\infty}(M,\Lambda^{p,q})\to C^{\infty}(M,\Lambda^{p,q+1})$. At each $p$, these maps form a complex of vector spaces

[TABLE]

and its reduced $L^{2}$-Dolbeault cohomology spaces are defined by:

[TABLE]

Since $\ker\bar{\partial}$ is a closed subspace in $L^{2}$, the reduced cohomology space $L^{2}\bar{H}^{p,q}(M)$ of our $G$-manifold is a Hilbert space with a strongly continuous action of $G$ by unitary transformations.

Let us consider $\bar{\partial}$ as the maximal operator in $L^{2}$ and let $\bar{\partial}^{\ast}$ be the Hilbert space adjoint operator (this requires the introduction of boundary conditions). We will deal with several constructions involving $\bar{\partial}$ and $\bar{\partial}^{*}$, which we define here. First, on the pre-domain $\mathcal{D}^{p,q}={\rm Dom\,}\bar{\partial}^{*}\cap C^{\infty}(\bar{M},\Lambda^{p,q})$, define the quadratic form

[TABLE]

The object dominating the frame in this work is Kohn’s Laplacian, defined by (on the $(p,q)$-forms)

[TABLE]

From the quadratic form $Q$ we may construct the operator $F=\bar{\partial}\bar{\partial}^{*}+\bar{\partial}^{*}\bar{\partial}+1$ on its (and $\square$’s) natural domain,

[TABLE]

where the closure is taken in the $Q$ norm, i.e. the graph norm of $\square$. The positivity of $\square$ implies that $F$ has a bounded inverse in $L^{2}(M)$.

Lemma 2.2**.**

The following orthogonal decompositions hold:

[TABLE]

In particular, we have a $G$-isomorphism of the spaces

[TABLE]

and the obvious consequence that if $\square$ is $G$-Fredholm in $L^{2}(M,\Lambda^{p,q})$, then

[TABLE]

Similar definitions and their consequences hold for the boundary complex, cf. Sect. V.4, [FK].

2.3. Sobolev spaces

We will have to describe smoothness of functions, forms, and sections of vector bundles using $G$-invariant Sobolev spaces which we describe here. The $G$ action induces an invariant Riemannian metric on $M$ so that with respect to this structure $M$ has bounded geometry. As in [Gro, S1] we may construct appropriate partitions of unity and, with local geodesic coordinates, assemble $G$-invariant integer Sobolev spaces $H^{s}(M)$. Fractional Sobolev spaces may be constructed by functional-analytic means or more specifically by interpolation.

Norms corresponding to negative Sobolev spaces are defined as usual;

[TABLE]

If $E$ is a vector $G$-bundle over $M$, then we may introduce a $G$-invariant inner product structure on $E$. Together with a $G$-invariant measure on $M$, we define the Hilbert spaces of sections of $E$ which we denote $H^{s}(M,E)$, for $s=0,1,2,\dots$. Because $X=M/G$ is compact, the spaces $H^{s}(M,E)$ do not depend on the choices of invariant metric on $M$ or of invariant inner product on $E$. Note that, in particular, spaces of sections in natural tensor bundles on a $G$-manifold have natural, invariant Sobolev structures.

We will also need anisotropic Sobolev spaces in neighborhoods of boundary points. Assume that our complex manifold $M$ has nonempty, smooth boundary $bM$, $\bar{M}=M\cup bM$, so that $M$ is the interior of $\bar{M}$, and ${\rm dim}_{\mathbb{C}}(M)=n$. Also assume that $\bar{M}\subset\tilde{M}$, where $\tilde{M}$ is a complex neighborhood of $\bar{M}$ of the same dimension, such that $bM$ is in the interior of $\tilde{M}$.

If $U$ is a sufficiently small neighborhood of a point $z\in bM$, then there exist real coordinates $x=(t_{1},\dots,t_{2n-1},\rho)$ in $U$ for which $z\leftrightarrow 0$, the set $U\cap M$ corresponds to $\{x\mid\rho<0\}$, and $bM$ corresponds to $\{x\mid\rho=0\}$, i.e. $\rho$ is the defining function. Coordinates such as these in $U$ are called a special boundary chart. By means of the Fourier transform in directions tangential to the boundary, it is possible to define tangential Sobolev norms $|||\cdot|||_{s}$, of arbitrary real order $s$, as follows, cf. Sect. II.4 of [FK].

With the tangential Fourier transform in a special boundary chart

[TABLE]

define for $s\in\mathbb{R}$, the operators

[TABLE]

(${\bf t}$ means tangential) and define the tangential Sobolev norms by

[TABLE]

With $D^{j}=D^{j}_{\bf t}=\frac{1}{i}\frac{\partial}{\partial t_{j}}$ for $j=1,\dots,2n-1$ the derivatives in tangential directions and $D^{2n}=D_{\rho}$, define the norms

[TABLE]

For two norms $\|\cdot\|$ and $|\cdot|$, we write $|\phi|\lesssim\|\phi\|$ to mean that there exists a constant $C>0$ such that $|\phi|\leq C\|\phi\|$ for $\phi$ in whatever set relevant to the context. Similarly, we will write $|\phi|\approx\|\phi\|$ to mean that $|\phi|\lesssim\|\phi\|$ and $\|\phi\|\lesssim|\phi|$.

3. Subelliptic Estimates

As in [K2], if $x\in\bar{M}$, we say that the $\bar{\partial}$-Neumann problem for $(p,q)$-forms satisfies a subelliptic estimate of order $\epsilon$ at $x$ if there exists a neighborhood $V$ of $x$ and a constant $C>0$ such that

[TABLE]

uniformly for $\phi\in C^{\infty}(V,\Lambda^{p,q})\cap{\rm Dom\,}\bar{\partial}\cap{\rm Dom\,}\bar{\partial}^{*}$.

Denote by $\mathcal{E}^{q}(\epsilon)$ the set of points $x$ of $\bar{M}$ for which there exists a neighborhood $U$ of $x$ on which a subelliptic estimate of order $\epsilon$ holds.

Definition 3.1**.**

We will say that a complex manifold $M$ satisfies a subelliptic estimate of order $\epsilon$ if each point of the boundary does.

Our setting will be a complex manifold $M$ with nonempty boundary that is also the total space of a $G$-bundle

[TABLE]

with $\bar{X}=\bar{M}/G$ compact, and satisfying the condition of the definition.

We will provide here an important fact concerning the relation between tangential and normal differentiability of functions in an elliptic system.

Lemma 3.2**.**

(Prop. 3.10, [K2]) Let $\epsilon>0$ and $x\in bM$. Then $x\in\mathcal{E}^{q}(\epsilon)$ if and only if there exists a neighborhood $V^{\prime}$ of $x$ and a constant $C^{\prime}>0$ such that

[TABLE]

Proof.

This proposition is a consequence of the fact that $bM$ is noncharacteristic with respect to $Q$ (which is elliptic). See the proofs of Thm. 2.4.8 and the expression 2.3.5 in [FK] or [CS], Lemmata 5.2.3–4. Roughly speaking, the ellipticity of $\square$ allows one to write the derivatives of $\phi$ in the direction normal to the boundary in terms of the tangential derivatives and other terms in $\square\phi$. The tangential derivatives of $\phi$ are of course controlled by the tangential norms, $|||\cdot|||_{\epsilon}$ and thus so are the normal ones. Note also that if we vary $\epsilon$, the size of the neighborhood $V$ in which the subelliptic estimate holds need not change; the constant can be changed. ∎

3.1. The basic estimate

For the following definitions, the reader is referred to [FK] for details. Let $\Pi_{p,q}$ be the projection onto the space of forms of type $(p,q)$ as in Eq. (2). For each $p\in bM$, the Levi form at $p$ is the Hermitian form on the $(n-1)$-dimensional space $(\Pi_{1,0}\mathbb{C}T_{p}M)\cap\mathbb{C}T_{p}bM$ given by

[TABLE]

A complex manifold $M$ satisfies condition $Z(q)$ if the Levi form has at least $n-q$ positive eigenvalues or at least $q+1$ negative eigenvalues at each point $p\in bM$. A strongly pseudoconvex complex manifold satisfies properties $Z(q)$ for $q=1,2,\dots,n$.

Let $\omega_{1},\dots,\omega_{n}$ be a local orthonormal basis for $C^{\infty}(\bar{M},\Lambda^{1,0})$ on the patch $U\subset\tilde{M}$ and let $L_{1},\dots,L_{n}$ be dual vector fields. Then if $\phi\in C^{\infty}(\bar{M},\Lambda^{1,0})$ has support in $U$, we may write $\phi=\sum_{IJ}\phi_{IJ}\omega^{I}\wedge\bar{\omega}^{J}$. In $U$ define the norm $E_{U}$ with

[TABLE]

Again by a partition of unity argument as in [Gro, S1], one may sum these local definitions and obtain a global norm via $E(\phi)^{2}=\sum_{j}E_{U_{j}}(\phi)^{2}$.

That the basic estimate is satisfied in $\mathcal{D}^{p,q}$ means that there exists a $C>0$ such that $E(\phi)^{2}\leq CQ(\phi,\phi)$ uniformly for $\phi\in\mathcal{D}^{p,q}$.

Hörmander in [H] showed that the condition $Z(q)$ is equivalent to the basic estimate. We will need only the forward implication, proven in the compact case in [FK], Theorem 3.2.10. The proof there relies on the compactness of $M$ but we will need only local estimates, as has been exploited in [E], and uniformity as guaranteed in our case by the structure of $M$, a $G$-manifold with compact quotient $M/G$. When we say “Suppose the basic estimate holds in $\mathcal{D}^{p,q}$,” we will mean that the estimate above holds locally.

Folland and Kohn in Theorem 2.4.4 prove (a stronger version of) the following:

Theorem 3.3**.**

For every $p\in bM$ there is a (small) special boundary chart $V$ containing $p$ such that

[TABLE]

It is here that one introduces the basic estimate (b.e.) as follows,

[TABLE]

yielding the local subelliptic estimate

[TABLE]

of order $\epsilon=1/2$ in a small special boundary chart $V$. For noncompact $M$ we will follow [E, P1] and only assume that the basic estimate holds locally.

3.2. Finite type

A subelliptic estimate holds for the $\bar{\partial}$-Neumann problem near a point in the boundary of a pseudoconvex domain if and only if the D’Angelo type at the point is finite, [C1, D]. In [K2] the condition of “finite ideal type” is shown to be sufficient for subellipticity. In [K3], Kohn conjectures this condition to be equivalent to the notion of finite type due to D’Angelo. Another property, called property $(P)$ (cf. [C2]), as well as McNeal’s property $(\tilde{P})$ [Mc], quantified in Herbig [He], give rise to subelliptic estimates. Michel and Shaw [MS] and Henkin, Iordan, Kohn, [HIK]

Straube achieved in [St] the removal of the assumption of smoothness of the boundary for property $(P)$. A collection of properties on the boundary guaranteeing subelliptic estimates are collected in [FS].

3.3. Other criteria

The results of [K2, DF] imply that a subelliptic estimate on $(p,q)$-forms is equivalent to the absence of germs of $q$-dimensional complex varieties in the boundary. Furthermore, these estimates always hold on any bounded pseudoconvex domain with real-analytic boundary.

4. The $\bar{\partial}$-Neumann Problem

4.1. Interior Estimates

First, modify $Q$ by adding a Sobolev 1-norm:

[TABLE]

It is obvious that for each $\delta>0$ we have that $\|\phi\|_{1}\lesssim Q^{\delta}(\phi,\phi)$, i.e. Gårding’s estimate holds, as long as $\phi$ is in the natural domain of $Q^{\delta}$, defined as for $Q$. It follows that if $F^{\delta}$ is the operator induced by $Q^{\delta}$, (Prop. 1.3.3, [FK]) then $F^{\delta}$ has regularity properties similar to those of $F$ in the interior of $M$, that is a (genuine) gain of two derivatives, cf. Theorem 2.2.9, [FK]. Estimates of this type will not be used in the regularization procedure as they depend strongly on $\delta>0$.

Instead, one shows that a genuine estimate for the $Q^{\delta}$ gaining one derivative can be made to hold uniformly in $\delta$. This is Theorem 2.3.4 of [FK] and it is of the following form. For $\delta>0$ let $\phi^{\delta}$ be the unique solution of $F^{\delta}\phi^{\delta}=\alpha$. Then the following estimates hold,

[TABLE]

In particular, if $\alpha$ is smooth, then $\phi^{\delta}$ is smooth and the constants $C_{k}$ do not depend on $\delta$.

4.2. First boundary a priori estimate

Lemma 4.1**.**

Suppose that in $\mathcal{D}^{p,q}$ a subelliptic estimate of order $\epsilon>0$ holds in a special boundary chart $V$. Let $\{\zeta_{k}\}_{0}^{\infty}$ be a sequence of real functions in $\Lambda^{0,0}_{0}(V\cap\bar{M})$ such that $\zeta_{k}=1$ on ${\rm supp}\ \zeta_{k+1}$. Then for each integer $k\geq 1$

[TABLE]

Proof.

An application of Lemma 3.2 converts the assumption to

[TABLE]

In Sect. 6.4 of [KN], it is shown that (9) implies the tangential estimates (8).∎

Corollary 4.2**.**

([E], Eq. 3.1) Suppose that in $\mathcal{D}^{p,q}$ a subelliptic estimate of order $\epsilon>0$ holds in a special boundary chart $V$. Then we obtain a priori estimates for each integer $k\geq 1$,

[TABLE]

Proof.

The estimate (8) implies the result because $||D\zeta_{k}\phi||_{k\epsilon-1}\approx|||D\zeta_{k}\phi|||_{k\epsilon-1}$ by Lemma 3.2 and $||D\zeta_{k}\phi||_{k\epsilon-1}\approx||\zeta_{k}\phi||_{k\epsilon}$ obviously. Next, $|||\zeta_{1}F\phi|||_{(k-2)\epsilon}\lesssim\|\zeta_{1}F\phi\|_{(k-2)\epsilon}$ since the smaller norm doesn’t differentiate in the normal direction and the tangential directions’ order is the same.∎

These boundary estimates can be summed with the interior estimates (7) (invoking uniformity) so that (10) holds for $\phi$ with support inside $M$, thus for each integer $k\geq 1$,

[TABLE]

4.3. Second boundary a priori estimate

This subsection is a small modification of the appendix in [P1] and Sect. 6.4 of [KN]. Our goal is a version of the $\epsilon=1/2$ estimate, Lemma 7.9 [P1],

[TABLE]

The interior estimates from subsection 4.1 will be used unchanged here.

We will systematically label sequences of real-valued, cutoff functions $(\zeta_{k})_{k}\subset C^{\infty}_{c}(M)$ such that $\zeta_{k}|_{{\rm supp}(\zeta_{k+1})}=1$ for $k=0,1,2,\dots$.

Lemma 4.3**.**

Let $r>0$, $U$ be a special boundary chart and let $\zeta,\zeta_{0},\zeta_{1}$ be real-valued functions in $C^{\infty}_{c}(U)$ with $\zeta_{1}=1$ on ${\rm supp}(\zeta)$ and $\zeta_{0}=1$ on ${\rm supp}(\zeta_{1})$. Then for $A=\zeta_{1}\Lambda_{\bf t}^{r}\zeta$ and for $A^{\prime}$ the formal adjoint of $A$ with respect to the inner product on $L^{2}(M)$,

[TABLE]

[TABLE]

uniformly for $\phi\in\mathcal{D}^{p,q}\cap\Lambda_{0}^{p,q}(U\cap\bar{M})$.

Proof.

These are simple consequences of the fact that the domain $\mathcal{D}^{p,q}$ of $\vartheta$ is preserved under the application of a cutoff function (cf. 2.3.2 of [FK]) and lemmata 2.4.2 and 2.4.3 of [FK] applied to $\zeta_{0}u$. ∎

If we assume further that $\phi\in{\rm Dom}(F)$, (cf. [FK], Prop. 1.3.5) we may write

[TABLE]

The following is our replacement of Lemma 7.9 of [P1], utilizing the subelliptic estimate, cf. [KN], p 472, [FK], Lemma 2.4.6. Note the similarity to Prop 3.1.11, [FK], proven with the aid of compactness.

Lemma 4.4**.**

Suppose that in $\mathcal{D}^{p,q}$ a subelliptic estimate of order $\epsilon>0$ holds in a special boundary chart $V$. Let $\{\zeta_{k}\}_{0}^{\infty}$ be a sequence of real functions in $\Lambda^{0,0}_{0}(V\cap\bar{M})$ such that $\zeta_{k}=1$ on ${\rm supp}\ \zeta_{k+1}$. Then for each integer $k\geq 1$,

[TABLE]

uniformly for $\phi\in{\rm Dom}(F)\cap\mathcal{D}^{p,q}$.

Proof.

Assuming the subelliptic estimate and noting that multiplication by $\zeta_{1}$ preserves $\mathcal{D}^{p,q}$, we have

[TABLE]

If we insert a real-valued cutoff function $\zeta_{0}$ equal 1 on the support of $\zeta_{1}$ and apply Lemma (4.3), to the form $\zeta_{0}u$ we have

[TABLE]

[TABLE]

Now, by the generalized Schwartz inequality, we have

[TABLE]

Now, for any $c>0$ there exists a $C>0$ sufficiently large so that

[TABLE]

thus

[TABLE]

and we have shown that the lemma is true for $k=1$. Assume the lemma true for $k-1$, i.e.

[TABLE]

We follow and modify where necessary the proof in [KN], citing intermediate results. According to Lemma 3.2, we may make the second replacement in our estimates,

[TABLE]

the first is obvious. Abbreviating $\Lambda_{\bf t}^{(k-1)\epsilon}=\Lambda$ and $A=\zeta_{1}\Lambda\zeta_{k}$, Kohn-Nirenberg derive their (6.6),

[TABLE]

Next, KN arrive at

[TABLE]

which they substitute into the estimate (16), obtaining

[TABLE]

[TABLE]

[TABLE]

The induction hypothesis reads

[TABLE]

so

[TABLE]

Since $|||\zeta_{1}F\phi|||_{(k-2)\epsilon}\leq\|\zeta_{1}F\phi\|_{(k-2)\epsilon}$ and $\|\zeta_{0}F\phi\|^{2}_{(k-3)\epsilon}\leq\|\zeta_{1}F\phi\|_{(k-2)\epsilon}$, we have that

[TABLE]

Again invoking Lemma 3.2, we replace $|||D\zeta_{k}\phi|||_{k\epsilon-1}$ with $\|\zeta_{k}\phi\|_{k\epsilon}$ and obtain the result. ∎

The following is Thm. 4.5 from [P1], adapted to our current situation.

Theorem 4.5**.**

Let $q>0$ and suppose a subelliptic estimate of order $\epsilon$ holds in a $G$-manifold $M$ with compact orbit space $M/G$ and invariant structures. Then, for every integer $k\geq 2$ we have an estimate

[TABLE]

holding uniformly for $\phi\in{\rm Dom}(\square)\cap C^{\infty}(\bar{M},\Lambda^{p,q})$.

Proof.

From Lemma 4.4, uniformity of the group invariance, the compactness of $M/G$, and again as in [Gro, S1] we may construct appropriate partitions of unity (of bounded multiplicity) and glue together the local a priori estimates (14) to obtain the global estimate. ∎

4.4. Genuine estimates

So far, all that we have shown assumes that $\phi$ is a smooth solution of $F\phi=\alpha$ and derives estimates involving the Sobolev norms of $\phi$ in terms of those of $\alpha$. It remains to show that if $\phi=F^{-1}\alpha$ now is the element of $L^{2}$ guaranteed by the bounded invertibility of $F$, that this $\phi$ has the same smoothness properties as predicted by the a prioris. I.e. it is not yet clear that these derivatives of $\phi$ exist. There are several ways to deduce this, e.g. in §4.1–2 of [KN] who perform an elliptic regularization.222In [FK], §II.5, the same method as in [KN] is used, but the theorem’s validity is restricted to the case of compact $M$ as its proof uses Rellich’s lemma. In [E], §3 a different method is proposed, valid, like ours, in the case of $M$ noncompact. The upshot is that from the first a priori estimate it follows that

Theorem 4.6**.**

Let $\alpha\in L^{2}(M,\Lambda^{p,q})$ and a subelliptic estimate of order $\epsilon$ hold in $\mathcal{D}^{p,q}$ Let $U$ be an open subset of $\bar{M}$ with compact closure, and $\zeta,\zeta_{1}\in C^{\infty}_{c}(U)$ with $\zeta_{1}|_{{\rm supp}(\zeta)}=1$. If $q>0$, $j$ is a nonnegative integer, and $\alpha|_{U}\in C^{\infty}(U,\Lambda^{p,q})$, then $\zeta(\square+1)^{-1}\alpha\in C^{\infty}(M,\Lambda^{p,q})$ and

[TABLE]

uniformly for $\alpha$.

Corollary 4.7**.**

Let $M$ satisfy a subelliptic estimate of order $\epsilon>0$ uniformly, let $U$ be an open subset of $\bar{M}$ with compact closure, and $\zeta,\zeta_{1}\in C^{\infty}_{c}(U)$ for which $\zeta_{1}|_{{\rm supp}(\zeta)}=1$. If $q>0$, $j$ is a nonnegative integer, and $\alpha|_{U}\in H^{j\epsilon}(U,\Lambda^{p,q})$, then $\zeta(\square+1)^{-1}\alpha\in H^{(j+2)\epsilon}(M,\Lambda^{p,q})$ and there exist constants $C_{j}>0$ so that

[TABLE]

Proof.

This is an extension of Prop. 3.1.1 from [FK]. It is a density argument applied to the real estimates above.∎

The following is a modified version of Cor. 4.3 from [P1].

Corollary 4.8**.**

Let $q>0$ and $\square=\int_{0}^{\infty}\lambda dE_{\lambda}$ be the spectral decomposition of the Laplacian in $L^{2}(M,\Lambda^{p,q})$. If $\delta>0$ and $P=\int_{0}^{\delta}dE_{\lambda}$ then ${\rm im\,}P\subset C^{\infty}(\bar{M},\Lambda^{p,q})$.

Proof.

We show that ${\rm im\,}P\subset H^{s}_{\rm loc}(M,\Lambda^{p,q})$ for all $s>0$ and invoke the Sobolev embedding theorem. Let $U,\zeta,\zeta_{1}$ be as in the previous theorem. Since ${\rm im\,}((\square+1)^{-1})={\rm Dom\,}(\square)$, the following is true. For every $\phi\in{\rm Dom\,}(\square)$ with $\alpha=\square\phi+\phi\in H^{j\epsilon}_{\rm loc}(M)$, we have $\phi\in H^{(j+2)\epsilon}_{\rm loc}(M)$ and the estimate (18) holds uniformly. Let $\phi\in{\rm im\,}P$. Applying the theorem with $j=0$, we have ${\rm im\,}P\subset H^{2\epsilon}_{\rm loc}(M,\Lambda^{p,q})$. Now assume $\phi\in{\rm im\,}P\subset H^{j\epsilon}_{\rm loc}(M,\Lambda^{p,q})$. Then

[TABLE]

We conclude that $\phi\in H^{(j+2)\epsilon}_{\rm loc}(M,\Lambda^{p,q})$ and so ${\rm im\,}P\subset H^{(j+2)\epsilon}_{\rm loc}(M,\Lambda^{p,q})$.∎

Corollary 4.9**.**

Let $q>0$ and $\square=\int_{0}^{\infty}\lambda dE_{\lambda}$ be the spectral decomposition of the Laplacian in $L^{2}(M,\Lambda^{p,q})$. If $\delta>0$ and $P=\int_{0}^{\delta}dE_{\lambda}$ then ${\rm im\,}P\subset H^{\infty}(M,\Lambda^{p,q})$.

Proof.

Let $\phi\in{\rm im\,}P$. By Cor. 4.8, $\phi\in C^{\infty}(\bar{M},\Lambda^{p,q})$ and so Thm. 4.5 holds. Since $\phi\in{\rm Dom\,}\square^{k}$, $k=1,2,3,\dots$, we have

[TABLE]

∎

We need the following fact regarding Sobolev spaces on manifolds with boundary. For $s>0$, denote by $H^{-s}(\bar{M})$ the dual space of $H^{s}(\bar{M})$. I.e. $H^{-s}(\bar{M})=(H^{s}(\bar{M}))^{\prime}$. Remark 12.5 of [LM] gives that if $M$ is a manifold with boundary and $s>0$, then $H^{-s}(\bar{M})$ consists of elements of $H^{-s}(\tilde{M})$ whose support is in $\bar{M}$.

Corollary 4.10**.**

Let $q>0$ and $\square=\int_{0}^{\infty}\lambda dE_{\lambda}$ be the spectral decomposition of the Laplacian in $L^{2}(M,\Lambda^{p,q})$. If $\delta>0$ and $P=\int_{0}^{\delta}dE_{\lambda}$ then $P:H^{-s}(\bar{M},\Lambda^{p,q})\to H^{s}(M,\Lambda^{p,q})$ for any positive integer $s$.

Proof.

In Lemma 4.9 we established that spectral projections $P$ of $\square$ take $L^{2}(M)$ to $H^{s}(M)$ for all $s>0$. It follows that $P$ can be extended so that $P:H^{-s}(\bar{M})\to L^{2}(M)$. Since $P^{2}=P$ on $H^{\infty}(M)\subset L^{2}(M)$, a dense subspace of all the $H^{s}(\bar{M})$, $(s\in\mathbb{R})$ we conclude that $P:H^{-s}(\bar{M})\to H^{s}(M)$ for all $s>0$. ∎

5. The $G$-Fredholm Property of $\square$

We will need a description of $G$-operators in terms of their Schwartz kernels. If $P\in\mathcal{B}(L^{2}(M))^{G}$, its kernel $K_{P}$ satisfies

[TABLE]

Thus $K_{P}$ descends to a distribution on the quotient $\frac{M\times M}{G}$. The measure taken on $\frac{M\times M}{G}$ is simply the quotient measure.

Lemma 5.1**.**

If $P:L^{2}(M)\to H^{\infty}(M)$ is a self-adjoint projection, then its Schwartz kernel $K_{P}$ is smooth.

Proof.

Since ${\bf y}\mapsto\delta_{\bf y}$ is a smooth function on $\bar{M}$ with values in $H^{-\infty}_{c}(\bar{M})$, the composition

[TABLE]

is jointly smooth. ∎

Lemma 5.2**.**

If $P\in\mathcal{B}(L^{2}(M))^{G}$ is a self-adjoint, invariant projection so that ${\rm im}(P)\subset H^{\infty}(M)$, then $K_{P}\in L^{2}(\frac{M\times M}{G})$.

Proof.

Fix ${\bf x}\in\bar{M}$. If $P:L^{2}(M)\to C^{\infty}(\bar{M})$, the closed graph theorem applied to $P$ implies $u\in L^{2}(M)\mapsto(Pu)({\bf x})\in\mathbb{C}$ is a bounded linear functional. The Riesz representation theorem then gives that there exists a function $h_{\bf x}\in L^{2}(M)$ so that

[TABLE]

Since $(Pu)({\bf x})=\int_{M}K_{P}({\bf x},{\bf y})u({\bf y})d{\bf y}$, and agrees with $\langle h_{\bf x},u\rangle$ when $u$ has compact support, $h_{\bf x}=K_{P}({\bf x},\ \cdot\ )$ almost everywhere. We conclude that for any ${\bf x}\in\bar{M}$, $\int_{M}|K_{P}({\bf x},{\bf y})|^{2}d{\bf y}<\infty$.

Now consider $\phi({\bf x})=\int_{M}|K_{P}({\bf x},{\bf y})|^{2}d{\bf y}$. The function $\phi$ is constant on orbits since the measure on $M$ is invariant;

[TABLE]

Thus $\phi$ descends to a function on $\bar{M}/G=\bar{X}$. Since the map from $\bar{M}$ to $H^{-\infty}_{c}(\bar{M})$ defined by ${\bf y}\mapsto\delta_{\bf y}$ is continuous, the composition

[TABLE]

is a continuous function $\bar{M}\to L^{2}(M)$. We may conclude that $\phi:\bar{X}\to\mathbb{R}_{+}$ is continuous. Denote by $\frac{d{\bf x}}{dt}$ the quotient measure on $X$. The compactness of $\bar{X}$ together with continuity of $\phi$ imply that $\int_{X}\phi({\bf x})\frac{d{\bf x}}{dt}<\infty$. Thus we have that $K_{P}\in L^{2}(\frac{M\times M}{G})$. ∎

Choosing a measurable global section $x$ in $M$ and representing points ${\bf x}\in M$, ${\bf x}\to(t,x)\in G\times X$, we obtain an isomorphism of measure spaces $(M,d{\bf x})\cong(G\times X,dt\otimes dx)$. Whenever $P\in\mathcal{B}(L^{2}(M))^{G}$ and $K_{P}\in L^{2}_{\rm loc}(M\times M)$, this isomorphism and the criterion for invariance allow a representation

[TABLE]

with $\kappa\in L^{2}_{\rm loc}(G\times X\times X).$

Lemma 5.3**.**

Let $P\in\mathcal{B}(L^{2}(M))^{G}$. Then ${\rm Tr}_{G}(P^{*}P)=\int_{\frac{M\times M}{G}}|K_{P}|^{2}$.

Proof.

Let $(\psi_{k})_{k}$ be an orthonormal basis for $L^{2}(X)$. In the decomposition $L^{2}(M)\cong\bigoplus_{k}L^{2}(G)\otimes\psi_{k}$, the invariant operator $P$ has a matrix representation $P\to[L_{h_{kl}}]_{kl}$. In terms of this, we compute ${\rm Tr}_{G}(P^{*}P)=\sum_{kl}\|h_{kl}\|_{L^{2}(G)}^{2}$.

Now, except on a set of measure zero, we have a description of $P$

[TABLE]

The distributional kernels $h_{ij}$ can be recovered from $\kappa$ by projecting into the summands in $L^{2}(M)\cong\bigoplus_{l}(L^{2}(G)\otimes\psi_{l})$,

[TABLE]

Let us compute the norm of $\kappa$ in $L^{2}(G\times X\times X)$. Since $(\psi_{j})_{j}$ is an orthonormal basis for $L^{2}(X)$, the set $(\bar{\psi}_{i}\otimes\psi_{j})_{ij}$ forms an orthonormal basis for $L^{2}(X\times X)$. By construction, $h_{ij}$ is equal the $ij^{th}$ Fourier coefficient of $\kappa$ with respect to the decomposition $L^{2}(G\times X\times X)\cong\bigoplus_{ij}(L^{2}(G)\otimes\psi_{i}\otimes\psi_{j})$. Hence $\sum_{ij}\|h_{ij}\|^{2}_{L^{2}(G)}=\|\kappa\|_{L^{2}(G\times X\times X)}^{2}$ and ${\rm Tr}_{G}(P^{*}P)=\|\kappa\|_{L^{2}(G\times X\times X)}^{2}=\int_{\frac{M\times M}{G}}|K_{P}({\bf x},{\bf y})|^{2}\ \frac{d{\bf x}d{\bf y}}{dt}$.∎

Corollary 5.4**.**

If $P\in\mathcal{B}(L^{2}(M))^{G}$ is an invariant self-adjoint projection such that ${\rm im}(P)\subset H^{\infty}(M)$, then ${\rm Tr}_{G}(P)<\infty$.

Remark 5.5*.*

All the previous results extend trivially to operators acting in bundles.

Theorem 5.6**.**

For $q>0$, the operator $\square$ on $M$ is $G$-Fredholm.

Proof.

Let $\square=\int_{0}^{\infty}\lambda dE_{\lambda}$ be the spectral decomposition of $\square$ and for $\delta>0$, $P=\int_{0}^{\delta}dE_{\lambda}$. Thus ${\rm im\,}(1-P)\subset{\rm im\,}\square$. Further, ${\rm im\,}P\subset L^{2}(M,\Lambda^{p,q})$ is closed, invariant and, by Corollary 4.9, ${\rm im\,}P\subset H^{\infty}(M,\Lambda^{p,q})$. Corollary 5.4 implies that ${\rm codim}_{G}({\rm im\,}(1-P))<\infty$. The requirement on the kernel of $\square$ is verified noting that $\ker(\square)\subset{\rm im\,}P$ in the above.∎

Corollary 5.7**.**

If $q>0$, $\dim_{G}L^{2}\bar{H}^{p,q}(M)<\infty$.

Proof.

By Lemma 2.2 $L^{2}\bar{H}^{p,q}(M)=\ker(\square_{p,q})={\rm im\,}(E_{0})$ which has finite $G$-dimension.∎

6. The $G$-Fredholm Property of $\square_{b}$

In Theorem 5.4.9, [FK] it is established that condition $Y(q)$ implies that

[TABLE]

with $\epsilon=1/2$. It follows that for the boundary Laplacian, all the results from the preceding discussions go through without change. More generally, it is sufficient for subelliptic estimates of the form above to hold, with and $\epsilon>0$. In [K2, K3], it is shown that both for domains and CR manifolds, finite ideal $q$-type implies that subellipticity holds for $(p,q)$-forms.

7. Applications

7.1. The $\bar{\partial}$- and $\bar{\partial}_{b}$-Neumann problems

The $G$-Fredholm property of $\square$ and $\square_{b}$ imply that we can sometimes solve the problems

[TABLE]

analogously to the compact case. Results from Sects. 2 and 3 of [P2] give conditions guaranteeing solvability of these equations in some closed, $G$-invariant subspaces of $L^{2}(M)$. For example, if $L$ is such a subspace of $L^{2}(M)$ containing a form with compact support (even only in one component), then by Cor. 2.3 and Lemmata 3.5, 3.6 of [P2], we have $\dim_{G}L=\infty$ and so ‘most’ elements $f\in L$ are good right-hand-sides for (19). Similarly, if $L$ contains an essentially unbounded function which has bounded $L^{2}(G)$-norms on slices, then also $\dim_{G}L=\infty$, by Lemma 2.4 of the same article.

7.2. Levi Problem

In [P2] we used the weaker results of [P1] to give sufficient conditions for the solution of the Levi problem on strongly pseudoconvex $M$. It would be interesting to see if the results of [BF] carry over to the present setting and whether the condition of amenability, introduced in [P2] remains of importance here.

8. Discrete cocompact group case

8.1. Criteria for finite $G$-dimensionality

In Lemmata 5.1, 5.2 we deduced the finite $G$-dimensionality of closed, invariant subspaces $L\subset L^{2}(M)$ assuming that $L\subset H^{s}(M)$ for arbitrarily large $s$. In the event that the group in question has a discrete, cocompact subgroup, we need much less. We have not yet been able to find an application for this fact as even in the subelliptic case we have the strong hypotheses satisfied, as in Corollary 4.10. Still, it seems of interest to ask whether the presence of such subgroups is important for the theorem to hold with the weaker hypotheses of $\epsilon$-gain or whether a proof exists that the theorem is true anyway. A possible characterization is in [MS], that of a central extension of a compact group. There the right- and left-invariant Sobolev spaces coincide.

Example 8.1*.*

Let us work out the analogue of Lemmata 5.1 and 5.2 completely and explicitly for an easy group. Let $G=\mathbb{R}$, and consider $L$, a closed, translation-invariant subspace of $L^{2}(\mathbb{R})$ satisfying a condition $\|u\|_{\epsilon}\leq C\|u\|_{0}$ for $u\in L$. As we have mentioned before, a projection $P:L^{2}(\mathbb{R})\to L^{2}(\mathbb{R})$ onto $L$ would take the form $u\mapsto Pu=h\ast u$ for some $h$, a distribution on $\mathbb{R}$. Taking the Fourier transform we find $\widehat{Pu}=\hat{h}\hat{u}$. Requiring $P=P^{2}$ implies that $h=h\ast h$, which translates to the condition that $\hat{h}$ satisfy $\hat{h}=\hat{h}^{2}$. In this case, $\hat{h}:\mathbb{R}\to\{0,1\}$, almost everywhere, so $h$ is completely characterized by $S={\rm supp}\ \hat{h}$. Let us now examine the estimate. By Plancherel’s theorem, the condition $\|u\|_{\epsilon}\leq C\|u\|_{0}$ for $u\in L$ is equivalent to $\|\Lambda^{\epsilon/2}\hat{u}\|_{0}\leq C\|\hat{u}\|_{0}$ for $u\in L$ where $\Lambda(\xi)=(1+|\xi|^{2})$. Concretely,

[TABLE]

or, involving $\hat{h}=\chi_{S}$, we can consider all $u\in L^{2}(\mathbb{R})$,

[TABLE]

Clearly this implies that $S=\{\xi\in\mathbb{R}\mid C^{2}-(1+|\xi|^{2})^{\epsilon}\geq 0\}$. Notice that

[TABLE]

Therefore, all $u\in L$ also satisfy $\|u\|_{\epsilon t}\leq C^{t}\|u\|_{0}\ \forall t\in\mathbb{R}$, with the constants’ growth identical to that in Lemma 8.5. We cannot invoke [KN, N] this time to obtain analyticity here as they assume compactness, but instead apply the Paley-Wiener theorem. This provides that $u\in L$ are analytic as their Fourier transforms have compact support, ${\rm supp}\ \hat{u}\subset{\rm supp}\ \hat{h}=S$ for all $u\in L$. We can also compute the $G$-dimension of $L$. We see that $L={\rm im}\ \check{\chi}_{S}\ast\cdot$, where $\check{}$ means the inverse Fourier transform. Applying Plancherel’s theorem again, we obtain $\|h\|^{2}=\|\check{\chi}_{S}\|^{2}=meas(S)=2\sqrt{C^{2/\epsilon}-1}$.

The following Paley-Wiener-type is from [GHS].

Lemma 8.2**.**

Let $L$ be a closed $\Gamma$-invariant subspace in $L^{2}(M,E)$, $L\subset W^{\epsilon}$ for some $\epsilon>0$ and there exists $C>0$ such that

[TABLE]

Then $\dim_{\Gamma}L<\infty$.

To prove this lemma [GHS] uses the following lemma about estimates of Sobolev norms on compact manifolds with boundary.

Lemma 8.3**.**

Let $X$ be a compact Riemannian manifold, possibly with a boundary. Let $E$ be a (complex) vector bundle with an hermitian metric over $\bar{X}$. Denote by $\langle\cdot,\cdot\rangle$ the induced Hermitian inner product in the Hilbert space $L^{2}(X,E)$ of square-integrable sections of $E$ over $X$. Denote by $W^{s}=W^{s}(X,E)$ the corresponding Sobolev space of sections of $E$ over $X$, $\|\cdot\|_{s}$ the norm in this space. Let us choose a complete orthonormal system $\{\psi_{j};\ j=1,2,\dots\}$ in $L^{2}(X,E)$. Then for all $\epsilon>0$ and $\delta>0$ there exists an integer $N>0$ such that

[TABLE]

Proof of Lemma 8.2. Let us choose a $\Gamma$-invariant covering of $M$ by balls $\gamma B_{k},\;k=1,\dots,m,\;\gamma\in\Gamma$, so that all the balls have smooth boundary (e.g. have sufficiently small radii). Let us choose a complete orthonormal system $\{\psi_{j}^{(k)}\;j=1,2,\dots\}$ in $L^{2}(B_{k},E)$ for every $k=1,\dots m$. Then $\{(\gamma^{-1})^{\ast}\psi_{j}^{(k)},\;j=1,2,\dots\}$ will be an orthonormal system in $\gamma B_{k}$ (here we identify the element $\gamma$ with the corresponding transformation of $M$).

Given the subspace $L$ satisfying the conditions in the Lemma let us define a map

[TABLE]

[TABLE]

Since $\dim_{\Gamma}L^{2}\Gamma\otimes\mathbb{C}^{mN}=mN<\infty$ the desired result will follow if we prove that $P_{N}$ is injective for large $N$. Assume that $u\in L$ and $P_{N}u=0$. Using Lemma 1.6 we get then

[TABLE]

where $\delta_{N}\to 0$ as $N\to\infty$ and $\|\cdot\|_{s,\gamma B_{k}}$ means the norm in the Sobolev space $W^{s}$ over the ball $\gamma B_{k}$. Summing over all $k$ and $\gamma$ we get

[TABLE]

where $C_{1}>0$ does not depend on $N$. This clearly contradicts (1.6) unless $u=0$. $\square$

Clearly this can be extended slightly using the results in [Ad].

Remark 8.4*.*

It is not necessary to require that $L$ is closed in $L^{2}$ in Proposition 1.5. For any $L$ satisfying (1.6) we can consider its closure $\bar{L}$ in $L^{2}$. Then obviously $\bar{L}\subset W^{\epsilon}$ and Proposition 1.5 implies that $\dim_{\Gamma}\bar{L}<\infty$.

It is important to note that for the discrete group, the $\epsilon$-gain in (20) is sufficient to obtain the finite $\Gamma$-dimensionality of $L$. Consider $\Gamma=\mathbb{Z}\subset\mathbb{R}=G$ and a function $h\in C^{\infty}_{c}(0,1)$. Clearly $\dim_{\Gamma}\langle h\rangle=1$ and an estimate like (20) certainly holds. Taking the bigger group, the Paley-Wiener theorem implies that $\dim_{G}\langle h\rangle=\infty$, each of the generated spaces $\langle h\rangle$ according to the appropriate groups, cf. [P2]. Sect. 2. This should be compared to the following fact. We have not been able to derive a similar property ($\epsilon$-gain implies finite-dimensionality) for continuous groups except for those that are central extensions of compact groups or reductive groups. Since all such groups have discrete cocompact subgroups [M] we gain nothing new.

8.2. Groups with Biinvariant Laplacians

It is well-known that Abelian groups and compact semisimple groups all have Laplacians (second-order elliptic differential operators) that commute with left- and right-translations, [M].

Lemma 8.5**.**

Let $G$ be a compact, semisimple Lie group, and suppose that $L$ is a right-invariant subspace of $L^{2}(G)$ satisfying

[TABLE]

It follows that $L$ is finite $G$-dimensional, finite-dimensional, and analytic.

Proof.

Since $G$ is compact, semisimple, it has a biinvariant elliptic Laplacian, $\Delta$. Since $L$ is right-invariant, $L$ can be written as the image of left-convolution by some distribution $h$ so that $L_{h}=L_{h}^{*}=L_{h}^{2}$. Abbreviate the convolution $L_{h}=h\ast\cdot$ and the invariant operator $\sqrt{1+\Delta}=\ ^{\prime}$.

Using the invariance of $\Delta$ and the identity $h=h\ast h$, it is easy to see that $h^{\prime}=h^{\prime}\ast h=h\ast h^{\prime}$. Furthermore, $h^{\prime\prime}=h^{\prime}\ast h^{\prime}$, and iterating, $h^{(k)}=(h^{\prime})^{\ast k}$. Now, using assumptions, identities, and the definition of the Sobolev norms,

[TABLE]

This means that the operator $L\ni u\mapsto h^{\prime}\ast u$ has norm less than or equal to $C$. Similarly (and perhaps surprisingly), the previous line shows that $\|h^{\prime}\ast u\|_{0}\leq C\|h\ast u\|_{0}$ for all $u\in L^{2}(G)$. So $h^{\prime}\ast u\in L^{2}(G)$, and this implies that $h^{\prime}=h^{\prime}\ast h$ acts on it in a bounded way:

[TABLE]

Iterating this process shows that $L_{h}u\in H^{\infty}(G)$ and so is smooth for all $u\in L^{2}(G)$. Therefore $h$ is smooth by Lemma 5.1 and ${\rm dim}_{G}\ {\rm Im}\ L_{h}=\int_{G}|h|^{2}=h(0)<\infty$. The finite-dimensionality of $L$ is just Rellich. Noting the exponential growth of the constants: $\|L_{h}u\|_{s}\leq C^{s}\|L_{h}u\|_{0}$ and the fact that the biinvariant Laplacian must have analytic coefficients (coming as is does from invariant vector fields) we conclude that $u\in{\rm im}\ L_{h}$ are in fact analytic, [N].∎

Remark 8.6*.*

With no change, the argument above works for a fractional (i.e. any $\epsilon>0$) gain $\|L_{h}u\|_{\epsilon}\leq C\|L_{h}u\|_{0}$, but it fails if one drops the assumption that the Laplacian be biinvariant.

The lemma has an obvious corollary.

Corollary 8.7**.**

If $L$ satisfies the hypotheses of the previous lemma and $u\in L$ is nonzero, then $\{x\in M\mid u(x)=0\}$ has no accumulation point.

Corollary 8.7 is reminiscent of the Paley-Wiener theorem in that the small support of the Fourier transform $\hat{u}$ translates to analyticity (implying support almost everywhere) of the function $u$. See [P2] for more about the Paley-Wiener theorem in this setting.

Remark 8.8*.*

Mimicking [A, GHS] it is possible to make a correspondence between compactness of an operator on a compact space and the $\Gamma$-Fredholm property of the lift of that operator on a regular covering.

9. Acknowledgments

The author thanks M. Engliš, G. Folland, and E. Straube for helpful conversations.