On noncompactness of the $\overline\partial$-Neumann problem on pseudoconvex domains in $\mathbb{C}^3$

TL;DR

This paper proves that certain smooth pseudoconvex domains in 3 with boundary containing a complex curve have a noncompact -Neumann operator, under finite type conditions, extending previous results.

Contribution

It establishes noncompactness of the -Neumann operator on (0,1)-forms for specific pseudoconvex domains in 3 with boundary complex manifolds and finite type assumptions.

Findings

Noncompact -Neumann operator on (0,1)-forms in 3 domains.

Boundary containing a 1-dimensional complex manifold implies noncompactness.

Finite D'Angelo 2-type condition is crucial for the result.

Abstract

In this paper we deal with the following question: is it true that any bounded smooth pseudoconvex domain in $\mathbb{C}^n$ whose boundary contains a $q$-dimensional complex manifold $M$ necessarily has a noncompact $\overline\partial$-Neumann operator $N_q$ ($1\leq q\leq n-1$)? We prove that a smooth bounded pseudoconvex domain $\Omega\subseteq\mathbb{C}^3$ with a one-dimensional complex manifold $M$ in its boundary has a noncompact Neumann operator on $(0,1)$-forms, under the additional assumption that $b\Omega$ has finite regular D'Angelo $2$-type at a point of $M$, improving previous results of Fu, \c{S}ahuto\u{g}lu, and Straube.

Equations

$$\rho (h(w))=\mathcal{O}(|w{|}^k)\forall w\in U,$$

$${\Omega }_1:={\Phi }_1(\Omega \cap V)=\{z\in U_1:\Re (z_3)>{\phi }_1(z_1,z_2,\Im (z_3))\}.$$

$$U:=\{z\in {\text{C}}^3:z_1\in D,(z_2,\Im (z_3))\in B,|\Re (z_3)|<t\},$$

$${\partial }_{z_2}^{j_0}{\partial }_{{\overline{z}}_2}^{k_0}\phi (z^{\ast },0,0)\ne 0.$$

$${\partial }_{z_2}^j{\partial }_{{\overline{z}}_2}^k{\phi }_m(z_1,0,0)=0\forall z_1\in D_m,\forall j,k:j+k\le m,$$

$${\partial }_{\Im (z_3)}{\phi }_m(z_1,0,0)=0\forall z_1\in D_m$$

$${\partial }_{z_2}^j{\partial }_{{\overline{z}}_2}^k{\phi }_m(z_1,0,0)=0\forall z_1\in D_m,\forall j,k:j,k\ge 1,j+k=m+1.$$

$${\phi }_{m+1}:D_{m+1}\times B_{m+1}\to (-t_{m+1},t_{m+1}),$$

$${\Phi }_{m+1}:U_m^{\prime }⟶U_{m+1}:=\{z\in {\text{C}}^3:z_1\in D_{m+1},(z_2,\Im (z_3))\in B_{m+1},|\Re (z_3)|<t_{m+1}\}$$

$$T_{\delta }(z_1,z_2,z_3):=(z_1,\delta z_2,{\delta }^{m+1}{z}_3)(\delta >0).$$

$$T_{{\delta }^{-1}}({\Omega }_m)=\{z\in T_{{\delta }^{-1}}(U_m):\Re (z_3)>{\delta }^{-m-1}{\phi }_m(z_1,\delta z_2,{\delta }^{m+1}\Im (z_3))\}.$$

$${\phi }_{m,0}(z_1,z_2,\Im (z_3))$$

$$\left\{z\in D_m\times {\text{C}}^2:\Re (z_3)>\sum _{{\gamma }_1+{\gamma }_2=m+1}\frac{1}{{\gamma }_1!{\gamma }_2!}{\partial }_{z_2}^{{\gamma }_1}{\partial }_{{\overline{z}}_2}^{{\gamma }_2}{\phi }_m(z_1,0,0)\cdot z_2^{{\gamma }_1}{\overline{z}}_2^{{\gamma }_2}\right\}$$

$$\sum _{{\gamma }_1+{\gamma }_2=m+1}\frac{1}{{\gamma }_1!{\gamma }_2!}{\partial }_{z_2}^{{\gamma }_1}{\partial }_{{\overline{z}}_2}^{{\gamma }_2}{\phi }_m(z_1,0,0)\cdot z_2^{{\gamma }_1}{\overline{z}}_2^{{\gamma }_2}\text{ is plurisubharmonic}.$$

$$\sum _{{\gamma }_1,{\gamma }_2\ge 1,{\gamma }_1+{\gamma }_2=m+1}\frac{1}{({\gamma }_1-1)!({\gamma }_2-2)!}{\partial }_{z_2}^{{\gamma }_1}{\partial }_{{\overline{z}}_2}^{{\gamma }_2}{\phi }_m(z_1,0,0)\cdot z_2^{{\gamma }_1-1}{\overline{z}}_2^{{\gamma }_2-1}\ge 0.$$

$$\sum _{{\gamma }_1+{\gamma }_2=m+1}\frac{1}{{\gamma }_1!{\gamma }_2!}{\partial }_{z_2}^{{\gamma }_1}{\partial }_{{\overline{z}}_2}^{{\gamma }_2}{\phi }_m(z_1,0,0)\cdot z_2^{{\gamma }_1}{\overline{z}}_2^{{\gamma }_2}$$

$$-{\left|\frac{1}{m!}{\partial }_{{\overline{z}}_1}{\partial }_{z_2}^{m+1}{\phi }_m(z_1,0,0)\cdot z_2^m\right|}^2.$$

$${\Phi }_{m+1}(z_1,z_2,z_3):=\left(z_1,z_2,z_3-\frac{2}{(m+1)!}{\partial }_{z_2}^{m+1}{\phi }_m(z_1,0,0)\cdot z_2^{m+1}\right).$$

$${\Phi }_{m+1}({\Omega }_m\cap U_m^{\prime })=\{z\in U_{m+1}:\Re (z_3)>{\phi }_{m+1}(z_1,z_2,\Im (z_3))\},$$

$${\phi }_{m+1}(z_1,z_2,\Im (z_3))$$

$${\partial }_{\Im (z_3)}{\phi }_{m+1}(z_1,0,0)$$

$${\partial }_{z_2}^j{\partial }_{{\overline{z}}_2}^k{\phi }_{m+1}(z_1,0,0)$$

$${\partial }_{z_2}^{m+1}{\phi }_{m+1}(z_1,0,0)$$

$${\partial }_{{\overline{z}}_2}^{m+1}{\phi }_{m+1}(z_1,0,0)$$

$${\partial }_{z_2}^{j_0}{\partial }_{{\overline{z}}_2}^{k_0}{\phi }_m(z^{\ast },0,0)\ne 0,$$

$${\partial }_{z_2}^j{\partial }_{{\overline{z}}_2}^k{\phi }_m(z_1,0,0)=0\forall z_1\in D_m,\forall j,k:j,k\ge 1,j+k=m+1,$$

$${\Omega }_{m+1},D_{m+1},B_{m+1},U_{m+1},{\phi }_{m+1}\text{ and }{\Phi }_{m+1}.$$

$${\Psi }_m({\Omega }_m)=\Omega \cap V_m,{\Omega }_m:=\{z\in U_m:\Re (z_3)>{\phi }_m(z_1,z_2,\Im (z_3))\},$$

$$H_m:=U_m\cap {\text{C}}^2\times \{0\}\hookrightarrow U_m\stackrel{{\Psi }_m}{⟶}{\text{C}}^3,$$

$${\rho }_m(z):={\phi }_m(z_1,z_2,\Im (z_3))-\Re (z_3),$$

$${\rho }_m\circ {\Psi }_m^{-1}\circ H_m(z_1,z_2)={\phi }_m(z_1,z_2,0)=\mathcal{O}(|z_2{|}^{m+1}).$$

Full text

Research supported by FWF-project P28154.

1. Introduction and main result

In this paper we deal with the following question, raised for instance in [FS01]: is it true that any bounded smooth pseudoconvex domain in $\C^{n}$ whose boundary contains a $q$-dimensional complex manifold $M$ necessarily has a noncompact $\dbar$-Neumann operator $N_{q}$ ($1\leq q\leq n-1$)? We refer the reader to [Str10] and [FS01] for the relevant background on the $\dbar$-Neumann problem in general, and the importance of compactness in particular.

Before stating our own contribution, let us list the main results presently known.

• (a)

Krantz discussed in [Kra88] the case of the bidisc in $\C^{2}$ (which strictly speaking is not smooth, but one may round the edges to fall under our assumptions). This is somehow the archetipal example for failure of compactness, since all the arguments known to establish it show first that the domain in consideration has a local “almost-product”, or more precisely “fibered”, structure (see Section 2 below, in particular the discussion after Lemma 2).

• (b)

The answer is affirmative in $\C^{2}$, as proved for general Lipschitz domains in [FS01]. The authors of that paper claim that this result has been part of the folklore for many years, with unpublished proofs by Catlin and Christ.

• (c)

Matheos exhibited a $2$-dimensional bounded smooth pseudoconvex Hartogs domain with no $1$-dimensional complex manifold in the boundary, which nonetheless has noncompact $\dbar$-Neumann operator (see [Mat98] or Theorem $4.25$ of [Str10]), disproving the reverse implication to the one we are considering. Christ and Fu (see [CF05]) later fully characterized compactness of $N_{1}$ on $2$-dimensional bounded smooth pseudoconvex Hartogs domains.

• (d)

Fu and Straube showed in [FS01] that on bounded locally convexifiable domains (in any dimension) $N_{q}$ is compact if and only if there are no $q$-dimensional complex manifolds in the boundary (see also section $4.9$ of [Str10]).

• (e)

Şahutoğlu and Straube proved failure of compactness of $N_{q}$ for smooth pseudoconvex domains in $\C^{n}$ with a $q$-dimensional manifold $M$ in the boundary, under the additional assumption that at a point of $M$ the boundary is strictly pseudoconvex in the directions transverse to $M$ (see [cSgS06] for the case $q=1$, and [Sah06] or Theorem $4.21$ of [Str10] for the general case). Notice that if $q=n-1$ this hypothesis is empty and the problem is completely solved.

The transversality condition in the last result is, in the words of one of the authors, “somewhat curious” as “one would expect a flatter boundary to be even more conducive to noncompactness” (p. 100 of [Str10]). Our goal in the present paper is to weaken significantly this assumption in the $3$-dimensional case (for $q=1$, since the case $q=2$ is settled), requiring only a finite-type condition in the directions transverse to the complex submanifold. More precisely, we prove the following:

Theorem 1**.**

Let $\Omega\subseteq\C^{3}$ be a bounded smooth pseudoconvex domain with a one-dimensional complex manifold $D$ in the boundary. Assume that there exists $P\in D$ such that the maximum order of contact of any $2$-dimensional complex manifold with $b\Omega$ at $P$ is bounded. Then the Neumann operator $N_{1}$ is not compact.

A few comments may help to clarify the scope of Theorem 1.

• (a)

The boundedness of the order of contact of complex hypersurfaces with $b\Omega$ at $P$ means that there exists a $k_{0}<+\infty$ for which the following holds: if $U$ is a neighborhood of [math] in $\C^{2}$, $h:U\rightarrow\C^{3}$ is a holomorphic mapping with Jacobian of rank $2$ at [math] such that $h(0)=P$, and

[TABLE]

then $k\leq k_{0}$. Here $\rho$ is any defining function of $\Omega$. This condition is called finite regular $2$-type by D’Angelo (see, e.g., [DK99]).

• (b)

Since $\Omega$ contains a one-dimensional complex manifold passing through $P$, our assumption is really a finite-type assumption in the directions transverse to $D$. See Lemma 3 below for a precise statement.

• (c)

The most dramatic way in which our assumption may be violated is if a $2$-dimensional complex manifold containing $D$ lies in the boundary, a condition equivalent to what is known as non-minimality of the boundary (at the point $P$) in CR geometry (see Theorem $1.5.15$ of [BER99]). Notice, however, that if this is the case the result of Şahutoğlu and Straube may be applied to conclude that $N_{2}$, the $\dbar$-Neumann operator on $(0,2)$-forms, fails to be compact, and this implies the non-compactness of $N_{1}$ too (Proposition $4.5$ of [Str10]).

• (d)

Another way to look at this result is to observe that to give a positive answer to the problem we stated at the beginning of the introduction, one would like to replace our assumption with compactness of $N_{2}$ (again because of the percolation of compactness up the $\dbar$-complex).

By results of Kohn [Koh72] and Greiner [Gre74], our hypothesis is instead equivalent to subellipticity at $P$ at the level of $(0,2)$-forms, which is slightly stronger.

• (e)

A hypothetical counterexample, i.e., a domain with a one-dimensional complex manifold $M$ in its boundary and compact Neumann operator on $(0,1)$-forms, must have the property that $M$ is made of minimal points satisfying an infinite-type condition transversally. Notice that the boundary of such a domain cannot be locally real-analytic around points of $M$.

1.1. Structure of the paper

The rest of the paper is devoted to the proof of Theorem 1. It follows the “plan” of [cSgS06] and [FS98], aiming to insert a smaller domain inside $\Omega$ that is tangent to $b\Omega$ along $D$ and has a product structure in appropriate coordinates. Morally, it is this product structure that causes the failure of compactness, but one must choose the smaller domain in such a way that its boundary has a sufficiently high order of contact with $b\Omega$, in order to transfer the non-compactness to the original domain $\Omega$. Under the assumptions of [cSgS06], the minimal order of contact $2$ is enough, and the construction boils down to a local structure lemma for domains with complex manifolds in the boundary which, in the $q=1$ case, follows from a result of Bedford and Fornæss (Lemma $1$ of [BFs81]). The key step and the main novelty in our proof is a refinement of this lemma in dimension $3$ (under our more general finite $2$-type assumption) that allows to increase this order of contact up to any finite value. Section 2 is devoted to its proof, that is a sort of parametric version of a normalization procedure used in the study of $2$-dimensional finite-type pseudoconvex domains (see, e.g., [Cat89]). It relies on an anisotropic scaling argument of a kind often used in harmonic and complex analysis. For the sake of clarity, we devote Appendices A and B to a rapid discussion of a few results we need in our proof.

The key technical tool to transfer non-compactness of $N_{1}$ from the smaller domain to the original one is a “noncompactness” lemma, of which we give in Section 3 the most appropriate formulation for our argument. The assumptions of the lemma involve the behaviour on the diagonal of the Bergman kernel of a domain and a tangent subdomain one dimension less than the one for which one wants to establish non-compactness. The main reason for the hypothesis in our theorem is that Bergman kernels are very well understood in dimension $2$ under a finite-type assumption, thanks to work of Catlin, as we recall in Section 4, while a general understanding lacks close to infinite type points and in higher dimension (despite significant progress: see, e.g., [MS94], [CD06], and [NP]).

In Section 5 we put all the ingredients together and complete the proof.

2. Local structure of a 3-dim. pseudoconvex domain with a one-dimensional complex manifold in the boundary

The starting point of the argument of [cSgS06] is the following local structure lemma for pseudoconvex domains whose boundary contains a $q$-dimensional complex manifold (Lemma $4.22$ of [Str10]).

Here and in what follows we employ the notation $\Re(z)$ and $\Im(z)$ for the real and imaginary parts of a complex variable $z$, and $\partial_{z}$ (resp. $\partial_{\overline{z}}$) for $\frac{\partial_{\Re(z)}-i\partial_{\Im(z)}}{2}$ (resp. $\frac{\partial_{\Re(z)}+i\partial_{\Im(z)}}{2}$).

Lemma 2**.**

Let $\Omega\subseteq\C^{n}$ be a smooth pseudoconvex domain with a $q$-dimensional complex manifold $M$ in its boundary. If $P\in M$ there exists a neighborhood $V$ of $P$ in $\C^{n}$ and a biholomorphism $\Phi:V\rightarrow\Phi(V)\subseteq\C^{n}$ such that:

1. (i)

$\Phi(P)=0$, 2. (ii)

$\Phi(M\cap V)=\Phi(V)\cap\{z_{q+1}=\cdots=z_{n}=0\}$, 3. (iii)

the exterior normal to $\Phi(\Omega\cap V)$ at points of $\Phi(V)\cap\{z_{q+1}=\cdots=z_{n}=0\}$ is given by the negative $\Re(z_{n})$-axis.

Before going further, let us rephrase the lemma in a somehow more explicit fashion in the $n=3$, $q=1$ case. Given $\Omega$ and $P$ as above there exist:

1. (1)

$t_{1}>0$, a disc $D_{1}\subseteq\C$ and a ball $B_{1}\subseteq\C\times\mathbb{R}$, both centered at [math], 2. (2)

a smooth function $\varphi_{1}:D_{1}\times B_{1}\rightarrow(-t_{1},t_{1})$ such that $\varphi_{1}$ and its gradient $\nabla\varphi_{1}$ vanish identically on $D_{1}\times\{0\}$, 3. (3)

a neighborhood $V$ of $P$ in $\C^{3}$ and a biholomorphism $\Phi_{1}$ of $V$ onto the set $U_{1}:=\{z\in\C^{3}\colon z_{1}\in D_{1},\ (z_{2},\Im(z_{3}))\in B_{1},\ |\Re(z_{3})|<t_{1}\}$ such that $\Phi_{1}(P)=0$ and

[TABLE]

In other words, in the appropriate local coordinates the domain is fibered over a disc and every fiber is a pseudoconvex domain in $\C^{2}$ “in normal form up to order $1$” (i.e., condition (2) above holds). Our next lemma improves this result, allowing to put simultaneously all the fibers in normal form up to higher order, under our finite regular $2$-type condition.

Lemma 3**.**

Let $\Omega\subseteq\C^{3}$ be a smooth pseudoconvex domain with a one-dimensional complex manifold $M$ in its boundary. Assume moreover that $b\Omega$ has bounded order of contact with $2$-dimensional complex manifolds at a point $P\in M$. Then there exist:

• •

a neighborhood $V$ of $P$ in $\C^{3}$,

• •

$t>0$, a disc $D\subseteq\C$ and a ball $B\subseteq\C\times\mathbb{R}$, both centered at [math],

• •

a biholomorphism $\Phi:V\longrightarrow U$, where

[TABLE]

such that $\Phi(P)=0$,

• •

a smooth function $\varphi:D\times B\rightarrow(-t,t)$ and an even $\tau\in\mathbb{N}\setminus\{0\}$,

such that

1. (i)

$\Phi(\Omega\cap V)=\left\{z\in U\colon\Re(z_{3})>\varphi(z_{1},z_{2},\Im(z_{3}))\right\}$, 2. (ii)

$\partial_{\Im(z_{3})}\varphi(z_{1},0,0)=0$* for every $z_{1}\in D$,* 3. (iii)

$\partial^{j}_{z_{2}}\partial^{k}_{\overline{z}_{2}}\varphi(z_{1},0,0)=0$* for every $z_{1}\in D$ and $j,k$ such that $j+k<\tau$,* 4. (iv)

there exist $z^{*}\in D$ and $j_{0},k_{0}\geq 1$ such that $j_{0}+k_{0}=\tau$ and

[TABLE]

The proof of Lemma 3 is based on an induction procedure, whose inductive step we isolate as a separate lemma.

Lemma 4**.**

For $m\geq 1$ let $\varphi_{m}:D_{m}\times B_{m}\rightarrow(-t_{m},t_{m})$ be a smooth function, where $t_{m}>0$, $D_{m}\subseteq\C$ is a disc and $B_{m}\subseteq\C\times\mathbb{R}$ is a ball, both centered at [math].

Set $U_{m}:=\{z\in\C^{3}\colon z_{1}\in D_{m},\ (z_{2},\Im(z_{3}))\in B_{m},\ |\Re(z_{3})|<t_{m}\}$ and assume that the domain $\Omega_{m}:=\{z\in U_{m}\colon\Re(z_{3})>\varphi_{m}(z_{1},z_{2},\Im(z_{3}))\}$ is pseudoconvex at boundary points where $\Re(z_{3})=\varphi_{m}(z_{1},z_{2},\Im(z_{3}))$. Assume also that

[TABLE]

[TABLE]

and that

[TABLE]

If $m$ is even, (3) is a consequence of (1) and (2).

Then there exist an open set $U_{m}^{\prime}\subseteq U_{m}$, $t_{m+1}>0$, a disc $D_{m+1}\subseteq\C$ and a ball $B_{m+1}\subseteq\C\times\mathbb{R}$, both centered at [math], a smooth function

[TABLE]

and a biholomorphism

[TABLE]

such that the following holds:

1. (i)

$\Phi_{m+1}(\Omega_{m}\cap U^{\prime}_{m})=\Omega_{m+1}:=\{z\in U_{m+1}\colon\Re(z_{3})>\varphi_{m+1}(z_{1},z_{2},\Im(z_{3}))\}$, 2. (ii)

$\partial^{j}_{z_{2}}\partial^{k}_{\overline{z}_{2}}\varphi_{m+1}(z_{1},0,0)=0\qquad\forall z_{1}\in D_{m+1},\quad\forall j,k\colon j+k\leq m+1$, 3. (iii)

$\partial_{\Im(z_{3})}\varphi_{m+1}(z_{1},0,0)=0\qquad\forall z_{1}\in D_{m+1}$.

Proof.

Consider the biholomorphic transformations

[TABLE]

Observe that

[TABLE]

We put $\varphi_{m,\delta}(z_{1},z_{2},\Im(z_{3})):=\delta^{-m-1}\varphi_{m}(z_{1},\delta z_{2},\delta^{m+1}\Im(z_{3}))$. Since $T_{\delta^{-1}}$ is biholomorphic, $T_{\delta^{-1}}(\Omega_{m})$ is pseudoconvex at points where $\Re(z_{3})=\varphi_{m,\delta}(z_{1},z_{2},\Im(z_{3}))$.

We now apply Proposition 9 of Appendix A with $f=\varphi_{m}$ and $\{D_{\delta}\}_{\delta>0}$, the dilations of $\C\times\mathbb{R}\equiv\mathbb{R}^{3}$ corresponding to the exponents $d_{1}=d_{2}=1$ and $d_{3}=m+1$, and $\ell=m+1$. Notice that $\beta\cdot d=\beta_{1}+\beta_{2}+(m+1)\beta_{3}<m+1$ means $\beta_{1}+\beta_{2}\leq m$ and $\beta_{3}=0$, so that the hypothesis of Proposition 9 are equivalent to hypothesis (1) above. We therefore have that $\varphi_{m,\delta}(z_{1},z_{2},\Im(z_{3}))$ converges pointwise, together with all its derivatives, to

[TABLE]

In the second line we used the fact that $d_{3}=m+1$ and hypothesis (2).

We can now invoke part (ii) of Proposition 10 of Appendix B to conclude that the domain

[TABLE]

is pseudoconvex at boundary points in the interior of $D\times\C^{2}$. Equivalently, by part (i) of the same proposition,

[TABLE]

Assume momentarily that $m$ is even. Then subharmonicity of the above function in the $z_{2}$ variable gives

[TABLE]

Since a nonnegative homogeneous polynomial of odd degree is identically [math], we conclude that $\partial^{\gamma_{1}}_{z_{2}}\partial^{\gamma_{2}}_{\overline{z}_{2}}\varphi_{m}(z_{1},0,0)$ vanishes for every $z_{1}\in D_{m}$ and $\gamma_{1},\gamma_{2}\geq 1$ such that $\gamma_{1}+\gamma_{2}=m+1$. This proves that hypothesis (3) is superfluous when $m$ is even, as stated. Therefore, independently of the parity of $m$, we have that

[TABLE]

is plurisubharmonic. An elementary computation then reveals that the determinant of the complex Hessian of the function above equals

[TABLE]

By plurisubharmonicity this quantity has to be nonnegative for every $z_{1}\in D_{m}$ and $z_{2}\in\C$, which is possible only if $\partial_{\overline{z}_{1}}\partial^{m+1}_{z_{2}}\varphi_{m}(z_{1},0,0)$ vanishes on $D_{m}$, that is, if $\partial^{m+1}_{z_{2}}\varphi_{m}(\cdot,0,0)$ is holomorphic.

We are now ready to define $\Phi_{m+1}$ as follows:

[TABLE]

By what we just proved, it is clear that $\Phi_{m}$ is a biholomorphism of $U_{m}$ onto its image. Moreover, it is clear that there is a smaller neighborhood of [math] $U^{\prime}_{m}\subseteq U_{m}$ such that

[TABLE]

where $U_{m+1}$ is as in the statement and

[TABLE]

An elementary application of the chain rule finally gives

[TABLE]

This concludes the proof. ∎

We are now ready to present the proof of Lemma 3.

Proof of Lemma 3.

We consider the following iteration scheme.

1. (1)

We initially set $m=1$ (recall how $\Omega_{1},D_{1},B_{1},U_{1},\varphi_{1}$ and $\Phi_{1}$ have been defined just before the statement of Lemma 3). 2. (2)

We are given $\Omega_{m},D_{m},B_{m},U_{m},\varphi_{m}$ and $\Phi_{m}$. If there exist $z^{*}\in D_{m}$ and $j_{0},k_{0}\geq 1$ such that $j_{0}+k_{0}=m+1$ and

[TABLE]

we stop the iteration. If the opposite holds, i.e.,

[TABLE]

go to step (3). 3. (3)

Lemma 4 (whose hypothesis are easily seen to be satisfied) gives us

[TABLE]

Increase $m$ by $1$ and go back to step (2).

We claim that this iteration scheme stops after finitely many steps. Let us first draw the consequences of the claim and then prove it.

If the iteration stops after $m$ steps, by Lemma 4 we have in particular that $m$ has to be odd. The mapping $\Psi_{m}:=\Phi_{1}^{-1}\circ\cdots\circ\Phi_{m}^{-1}$ is well-defined on $U_{m}$ and it is a biholomorphism onto its image $V_{m}$ such that $\Psi_{m}(0)=P$ and

[TABLE]

where $\partial^{j}_{z_{2}}\partial^{k}_{\overline{z}_{2}}\varphi_{m}(z_{1},0,0)=0$ for every $z_{1}\in D_{m}$ and every $j,k$ such that $j+k\leq m$, but $\partial^{j_{0}}_{z_{2}}\partial^{k_{0}}_{\overline{z}_{2}}\varphi_{m}(z^{*},0,0)\neq 0$ for at least a point $z^{*}\in D_{m}$ and a couple of exponents $j_{0},k_{0}\geq 1$ such that $j_{0}+k_{0}=m+1$.

Setting $t=t_{m},D=D_{m},B=B_{m},U=U_{m},V=V_{m},\varphi:=\varphi_{m},\Phi=\Psi_{m}^{-1}$ and $\tau=m+1$, we have the thesis.

We are left with proving that the iteration actually halts. We argue by contradiction, assuming that it never stops. In this case, we can define $\Psi_{m}$ and $V_{m}$ as above for every $m$ and consider the composition

[TABLE]

that is a holomorphic mapping with Jacobian of rank 2 (because $\Psi_{m}$ is a biholomorphism) such that $H_{m}(0,0)=P$. Observe that $\rho_{m}\circ\Psi_{m}^{-1}$ is a local defining function for $\Omega$ on $V_{m}$, if

[TABLE]

and that

[TABLE]

Since $m$ is arbitrary, this contradicts the finite-type hypothesis and completes the proof.

∎

3. A non-compactness lemma

Special instances of the following lemma play a key role both in [FS98] and in [cSgS06], although it does not appear as a separate statement in these papers. The strictly pseudoconvex version of the result can be found in [Str10] (Lemma $4.23$). We will need a more general statement, whose short proof is obtained adapting the arguments of the cited papers.

Lemma 5**.**

Let $\Omega_{2}\subseteq\Omega_{1}\subseteq\C^{n}$ be domains that share a boundary point $P$. Denote by $K_{1}$ (resp. $K_{2}$) the Bergman kernel of $\Omega_{1}$ (resp. $\Omega_{2}$) and assume that:

1. (i)

there exists a neighborhood $V$ of $P$ in $\C^{n}$ such that $K_{1}(z,\cdot)$ is bounded on $\Omega_{1}\cap V$ for every $z\in\Omega_{1}\cap V$, 2. (ii)

there exists a sequence $\{P_{n}\}_{n}\subseteq\Omega_{2}$ converging to $P$ such that

[TABLE]

Then the restriction operator $A(\Omega_{1})\rightarrow A(\Omega_{2})$ is not compact.

The symbol $\lesssim$ means that the implicit constant is independent of $n$.

Before going to the proof, a few comments are in order.

1. (a)

An effective way to verify assumption (i) is via the observation (see [Ker72]) that, since $K_{1}$ is holomorphic in its first variable,

[TABLE]

Here $\chi_{z}$ is an $L^{1}$-normalized real-valued radial test function centered at $z$ and with support inside $\Omega_{1}$, while $B_{1}$ is the Bergman projection of $\Omega_{1}$. Therefore, all we need for (i) to hold is that $B_{1}$ maps test functions on $\Omega_{1}\cap V$ to functions that are bounded on the same region, a rather weak regularity requirement, which is certainly satisfied if $\Omega_{1}$ is pseudoconvex, smooth in a neighborhood of $P$, and has a compact $\dbar$-Neumann operator. In fact under these assumptions a well-known theorem of Kohn and Nirenberg states that $N_{1}$, the $\dbar$-Neumann operator, maps smooth $(0,1)$-forms to forms that are smooth up to the boundary in a neighborhood of $P$ (see Theorem 4.6 of [Str10], where the assumption is that the domain is everywhere smooth, but the argument is local). Combining this with Kohn’s identity $B_{1}(\chi_{z})=\chi_{z}-\dbar^{*}N_{1}\dbar\chi_{z}$, the conclusion follows immediately. 2. (b)

The condition $\lim_{n\rightarrow+\infty}K_{1}(P_{n},P_{n})=+\infty$ holds for any sequence $\{P_{n}\}_{n}$ approaching a boundary point of $\Omega_{1}$ that satisfies the outer cone condition (see [JP13]), which is the case if $\Omega_{1}$ is smooth in a neighborhood of $P$. 3. (c)

The inequality $K_{2}(z,z)\geq K_{1}(z,z)$ follows from the inclusion $\Omega_{2}\subseteq\Omega_{1}$ and the variational characterization

[TABLE]

Thus the key assumption of Lemma 5 is the bound $K_{2}(P_{n},P_{n})\lesssim K_{1}(P_{n},P_{n})$. Observe that in view of (4), the larger $\Omega_{2}$ is, the easier it is for it to satisfy this condition.

Proof.

We argue by contradiction, assuming that the restriction operator is compact. Consider the sequence of functions

[TABLE]

By elementary properties of the Bergman kernels, they are holomorphic and satisfy $\int_{\Omega_{1}}|f_{n}|^{2}=1$. Using our assumption and passing to a subsequence, we can assume that $f_{n}$ converges in $L^{2}(\Omega_{2})$ and pointwise almost everywhere to a function $g\in A(\Omega_{2})$.

Moreover, by (i) and the first half of (ii), we immediately have that

[TABLE]

Hence $g$ vanishes almost everywhere on $V\cap\Omega_{2}$. Being holomorphic, it vanishes everywhere on $\Omega_{2}$.

Now consider $K_{1}(\cdot,P_{n})$ as an element of $A(\Omega_{2})$. The reproducing formula for $\Omega_{2}$ gives

[TABLE]

by Cauchy-Schwartz and the second half of (2). Rearranging this inequality, we find $\int_{\Omega_{2}}|f_{n}|^{2}\gtrsim 1$, which clearly contradicts the fact that $f_{n}$ converges to [math] in $L^{2}(\Omega_{2})$. ∎

4. Bergman kernels on the diagonal on domains of finite type in $\C^{2}$

We recall a result of Catlin about Bergman kernels of smooth pseudoconvex domains of finite-type in $\C^{2}$.

Theorem 6**.**

Let $\Omega\subseteq\C^{2}$ be a smooth pseudoconvex domain. Assume that $P\in b\Omega$ is a point of type $\tau<+\infty$ and denote by $\nu$ the exterior normal to $\Omega$ at $P$. Let $V$ be a bounded neighborhood of $P$ in $\C^{2}$ and $K$ the Bergman kernel of $\Omega\cap V$. Then there exists $\delta_{0}>0$ such that

[TABLE]

This is Theorem $2$ of [Cat89], except that it deals with the Bergman kernel of a bounded smooth pseudoconvex domain. To obtain the result stated above, one can apply the localization lemma of Ohsawa [Ohs84].

Next, we want to state an elementary upper bound for the Bergman kernel of a non-smooth model domain “of type $\tau$”, that will be needed later. The proof is standard.

Proposition 7**.**

Consider the domain

[TABLE]

where $\tau,C>0$. Let $W$ be a neighborhood of the origin in $\C^{2}$ and $K$ be the Bergman kernel of $\Omega_{\tau,C}\cap W$ . Then there exists $\delta_{0}>0$ such that

[TABLE]

Proof.

Let us check that the bidisc $E_{\delta}:=D(0,\delta^{\frac{1}{\tau}})\times D((2C+1)\delta,\delta)$ is contained in $\Omega_{m,C}$ for every $\delta\in(0,1)$. If $(z_{1},z_{2})\in E_{\delta}$, $|\Im(z_{2})|<\delta$ and

[TABLE]

Since $\Re(z_{2})>2C\delta$, if $\delta<\delta_{0}$ (with $\delta_{0}$ dependent on $W$) we have $(z_{1},z_{2})\in\Omega_{\tau,C}$.

Let now $f$ be a holomorphic function on $\Omega_{\tau,C}\cap W$ having $L^{2}$ norm equal to $1$. Since holomorphic functions are pluriharmonic, we can estimate (for $\delta<\delta_{0}$ depending on $W$)

[TABLE]

where we used the mean value property in both coordinates and Jensen’s inequality. Taking the supremum over $f\in A(\Omega_{\tau,C}\cap W)$ of norm $1$ and computing the volume we conclude that

[TABLE]

Rescaling the variable $\delta$, we obtain the stated estimate. ∎

5. Proof of Theorem 1

We argue by contradiction, assuming that $N_{1}$ is compact.

We begin by applying Lemma 3 and keeping the notation of its statement. We may assume that the neighborhood $V$ of the point $P$ is strictly pseudo-convex (for example, a ball), if we replace $U$ with a smaller open set.

An anisotropic Taylor expansion (Proposition 8 of Appendix A), together with the conclusions (ii) and (iii) of Lemma 3, gives

[TABLE]

where $C$ and the implicit constant of the big O are uniform in $z_{1}\in D$. Thus

[TABLE]

where $D^{\prime}\subseteq D$ is a small disc centered at [math],

[TABLE]

for some $\varepsilon>0$, and

[TABLE]

We claim that the two-dimensional domains

[TABLE]

satisfy the hypothesis of Lemma 5. Here $z^{*}$ is as in the statement of Lemma 3, and $P=(z^{*},0,0)$ is the common boundary point. Conditions $(ii)$, $(iii)$, and $(iv)$ of Lemma 3 imply that $P$ is point of type $\tau$ of $\Omega_{1}$, and that the exterior normal at $P$ is $\nu=(0,-1)$. Thus Theorem 6 and Proposition 7 combine to give the key hypothesis (the second half of $(ii)$ of Lemma 5). For the other assumptions recall that, by our observations after the statement of the lemma, all we need is the compactness of the $\dbar$-Neumann operator of $\Omega_{1}$. To see this, notice that this domain is the intersection of a $2$-dimensional smooth bounded pseudoconvex domain of finite-type with $\{(z_{2},z_{3})\in\C^{2}\colon(z^{*},z_{2},z_{3})\in U\}$. The latter set is a slice of $\Phi(V)$, which is strictly pseudoconvex thanks to our choice of $V$. Since compactness localizes to such intersections (see part (2) of Proposition $4.4$ in [Str10]), our goal follows from a theorem of Catlin (see [Cat84]).

Now the proof follows closely the arguments of [FS98] and [cSgS06]. Lemma 5 yields a bounded sequence $\{f_{n}\}_{n}\subseteq A(\Omega_{1})$ such that no subsequence converges in $L^{2}(\Omega_{2})$. The Ohsawa-Takegoshi extension theorem (see [OT87], or Theorem $2.17$ of [Str10]) gives $\{F_{n}\}_{n}\subseteq A(\widetilde{\Omega})$ such that

[TABLE]

The $(0,1)$-forms $F_{n}d\overline{z}_{1}$ are closed and uniformly bounded in $L^{2}(\widetilde{\Omega})$, and so are their pull-backs $\Phi^{*}(F_{n}d\overline{z}_{1})$ (the biholomorphism $\Phi:V\rightarrow U^{\prime}$ may be assumed to be smooth up to the boundary to guarantee this), which are defined on $\Omega\cap V$. The latter is a domain (again taking $V$ small enough) whose $\dbar$-Neumann operator on $(0,1)$-forms is compact. In fact we assumed, by contradiction, that this is the case for the whole domain $\Omega$, and we can localize thanks to Proposition $4.4$ of [Str10] as before, because $V$ is strictly pseudoconvex. It is an elementary fact that the compactness of the $\dbar$-Neumann operator implies the compactness of the canonical solution operator (see, e.g., Proposition $4.2$ of [Str10]), so that composing with $\Phi^{-1}$ and passing to a subsequence we get a Cauchy sequence $\{g_{n}\}_{n}\subseteq L^{2}(\widetilde{\Omega})$ with the property that $\dbar g_{n}=F_{n}d\overline{z}_{1}$. In particular $\partial_{\overline{z_{1}}}g_{n}=F_{n}$.

If $(z_{2},z_{3})\in\Omega_{2}$ and $\eta$ is a radial real-valued test function on $D^{\prime}$ such that $\int\eta=1$, we have the bound

[TABLE]

where in the second line we used the inclusion (5) and the mean-value property for holomorphic functions, and in the third line an integration by parts. This proves that $\{f_{n}\}_{n}$ is Cauchy in $L^{2}(\Omega_{2})$, the contradiction we seeked. Theorem 1 is proved.

Appendix A Anisotropic scaling

Given $d\in(\mathbb{N}\setminus\{0\})^{q}$, we consider the associated one-parameter group of anisotropic dilations of $\mathbb{R}^{q}$

[TABLE]

It is useful to introduce a corresponding anisotropic norm $N(y):=\max_{j=1,\dots,q}|y_{j}|^{\frac{1}{d_{j}}}$, that satisfies $N(D_{\delta}y)=\delta N(y)$ ($\forall\delta>0$). Our first elementary technical tool is an anisotropic Taylor series.

Proposition 8**.**

If $g$ is a smooth function defined in a neighborhood of [math] in $\mathbb{R}^{q}$,

[TABLE]

where the constant implicit in the big O depends on an upper bound on finitely many derivatives of $g$ in a neighborhood of [math].

Proof.

The ordinary Taylor expansion is

[TABLE]

where $|\gamma|=\sum_{j=1}^{q}\gamma_{j}$ and the constant implicit in the big O depends on an upper bound on finitely many derivatives of $g$ in a neighborhood of [math]. Hence,

[TABLE]

where in the first identity we used the fact that $d\cdot\gamma$ takes only integer values. ∎

We can now state and prove a useful proposition about anisotropic scaling of functions.

Proposition 9**.**

Let $B_{1}\subseteq\mathbb{R}^{p}$ and $B_{2}\subseteq\mathbb{R}^{q}$ be open euclidean balls centered at [math], and let $f:B_{1}\times B_{2}\rightarrow\mathbb{R}$ be a smooth function. Let $d\in(\mathbb{N}\setminus\{0\})^{q}$ and $\{D_{\delta}\}_{\delta>0}$ the associated one-parameter group of dilations.

If $\partial_{y}^{\beta}f(x,0)=0$ for every $x\in B_{1}$ and for every $\beta\in\mathbb{N}^{q}$ such that $d\cdot\beta<\ell$, then

[TABLE]

where we adopted the usual multi-index notation. Notice that the rescaled function in the left-hand side is defined on $B_{1}\times D_{\delta^{-1}}B_{2}$, so that the limit is defined for every $(x,y)\in B_{1}\times\mathbb{R}^{q}$.

Moreover, the pointwise convergence holds also for derivatives, i.e., setting

[TABLE]

we have

[TABLE]

Proof.

Observe that $\partial_{x}^{\alpha}\partial_{y}^{\beta}f_{\delta}(x,y)=\delta^{-\ell+d\cdot\beta}\partial_{x}^{\alpha}\partial_{y}^{\beta}f(x,D_{\delta}y)$ and recall that by assumption $\partial_{y}^{\gamma}\partial_{x}^{\alpha}\partial_{y}^{\beta}f(x,0)=0$ for every $x$ and every $\gamma$ such that $d\cdot\gamma<\ell-d\cdot\beta$. Proposition 8 yields

[TABLE]

Notice that the first summand may be empty (when $d\cdot\beta>\ell$). It is easy to see that the first term above equals $\partial_{x}^{\alpha}\partial_{y}^{\beta}f_{0}(x,y)$. Letting $\delta$ tend to [math], we get the thesis. ∎

Appendix B Pseudoconvexity and defining functions

Fix a neighborhood $V$ of the origin in $\C^{n-1}\times\mathbb{R}$ and a constant $t>0$. We define

[TABLE]

If $\varphi:V\rightarrow\mathbb{R}$ is a $C^{2}$ function, we consider the domain

[TABLE]

We want to express in terms of $\varphi$ the condition of pseudoconvexity of $\Omega$ at points where $\Re(z_{n})=\varphi(z^{\prime},\Im(z_{n}))$. It is clear that $b\Omega\cap U$ is parametrized by

[TABLE]

The complex tangent $T^{(1,0)}_{z}b\Omega$ at the point corresponding to $(z^{\prime},y)$ is the set of $u\in\C^{n}$ such that

[TABLE]

as one can compute using the defining function $\rho(z)=\varphi(z^{\prime},\Im(z_{n}))-\Re(z_{n})$.

Since $1+i\partial_{y}\varphi(z^{\prime},y)$ does not vanish on $V$, we can parametrize $T^{(1,0)}_{z}b\Omega$ by

[TABLE]

Computing the second order derivatives of $\rho$ we can finally express the pseudoconvexity of $b\Omega$ on $U$ as the inequality

[TABLE]

that must hold for every $u\in\C^{n-1}$ and $(z^{\prime},y)\in V$.

From this inequality one can easily deduce the following proposition.

Proposition 10**.**

1. (i)

If $\varphi(z^{\prime},y)\equiv\varphi(z^{\prime})$ is independent of $y$, the pseudoconvexity of $\Omega$ is equivalent to the plurisubharmonicity of $\varphi(z^{\prime})$. 2. (ii)

If $\{\varphi_{k}\}_{k\in\mathbb{N}}$ is a sequence of $C^{2}$ real-valued functions as above such that the corresponding domains $\Omega_{k}$ are all pseudoconvex (at points where $\Re(z_{n})=\varphi_{k}$), and if $\varphi_{k}$ converges pointwise together with all the derivatives up to order $2$ to a $C^{2}$ function $\varphi$, then the domain $\Omega$ associated to $\varphi$ is also pseudoconvex (at points where $\Re(z_{n})=\varphi$).

Acknowledgements

The author would like to thank the people of the complex analysis group of the University of Vienna for their comments and observations on his work, in particular Friedrich Haslinger, Bernhard Lamel, and Michael Reiter. The author’s research have been supported by the FWF-project P28154.