Positivity of Brown-York mass with quasi-positive boundary data

TL;DR

This paper proves the positivity of the Brown-York mass under quasi-positive boundary conditions, extending previous results, and establishes a related rigidity theorem.

Contribution

It introduces a generalized positivity result for Brown-York mass with quasi-positive boundary data and proves a corresponding rigidity theorem.

Findings

Positivity of Brown-York mass under quasi-positive boundary data

Extension of previous positivity results

Rigidity theorem for quasi-positive boundary conditions

Abstract

In this short note, we prove positivity of Brown-York mass under quasi-positive boundary data which generalize some previous results by the authors. The corresponding rigidity result is obtained.

Equations

$${\mathfrak{m}}_{BY}({\Sigma }_{\ell };\Omega ,g)\ge 0$$

$${\int }_{\Sigma }{H}_{0}d\sigma$$

$${\mathfrak{m}}_{BY}(\Sigma ;\Omega ,g)=\frac{1}{8\pi }{\int }_{\Sigma }(H_0-H)d\sigma .$$

$$\left\{\begin{array}[]{ll}8\Delta_{g}u-Ru=0\ \ \text{in $\Omega$}\\ u=1\ \ \text{on $\partial\Omega$.}\end{array}\right.$$

$${\mathfrak{m}}_{BY}(\Sigma ;\Omega ,g)>0.$$

$$\left\{\begin{array}[]{rcl}\Delta_{g}u&=&0\ \ \mathrm{in}\ \Omega\\ u&=&1-\epsilon\phi\ \ \mathrm{on}\ \partial\Omega.\end{array}\right.$$

$$H_{g_1}=\frac{1}{u^2}\left(H_g+\frac{4}{u}\frac{\partial u}{\partial \nu }\right)$$

$${\int }_{{\Sigma }_2}(H_0-H_{g_3})d{\sigma }_{g_3}\ge 0.$$

$${\int }_{\partial M_4}{H}_{h}d{\sigma }_h>{\int }_{\partial M_4}{H}_{g_4}d{\sigma }_{g_4}=2{\int }_{\Sigma }{H}_{g_3}d{\sigma }_{g_3}.$$

$$2{\int }_{\Sigma }{H}_{0}d\sigma \ge {\int }_{\partial M_4}{H}_{h}d\sigma \ge 2{\int }_{\Sigma }{H}_{g_3}d{\sigma }_{g_3}.$$

$${\mathfrak{m}}_{BY}(\Sigma ;\Omega ,g)\ge 0.$$

$$\left\{\begin{array}[]{rcl}\Delta_{g}u&=&0\ \ \mathrm{in}\ \Omega\\ u&=&1-\epsilon\phi\ \ \mathrm{on}\ \partial\Omega.\end{array}\right.$$

$$K(ϵ)>-{\kappa }^2(ϵ)$$

$$\{|x|<{\kappa }^{-2}(ϵ)\}$$

$${\int }_{\Sigma }(H_{\kappa (ϵ)}-H(ϵ))\cosh (\kappa (ϵ)r)d{\sigma }_{g(ϵ)}\ge 0$$

$$\underset{i\to \infty }{\mathrm{liminf}}{\int }_{\Sigma }(H_{\kappa ({ϵ}_i)}-H_g)d\sigma \ge 0.$$

$$\underset{i\to \infty }{\lim }{\int }_{\Sigma }{H}_{\kappa ({ϵ}_i)}d\sigma ={\int }_{\Sigma }{H}_{0}d\sigma .$$

$${\mathfrak{m}}_{BY}(\Sigma ;\Omega ,g)\ge 0.$$

$${\mathfrak{m}}_{BY}(\Sigma ;\Omega ,g)>0.$$

$$|\nabla u_r|(x)\le \underset{\partial B_r(p)\cap B_{\rho }(x)}{\sup }{H}_{+}+\frac{C}{\rho },$$

$$Rc\ge -\frac{1}{1000r^2}\text{in}{B}_r(x),$$

$$|\nabla u_r|(x)\le \frac{C}{d(x,p)},$$

$$|\nabla u|(x)\le \frac{C}{d(x,p)},$$

$$|\partial E{|}_g\le |\partial F{|}_g\le |\partial E{|}_g+\theta ;{\mathfrak{m}}_H(\partial F)\ge {\mathfrak{m}}_H(\partial E)-\theta >0.$$

$${\mathfrak{m}}_{ADM}(g_{ϵ})\le ϵ.$$

$${\tilde{R}}_{\tau }u-8{\Delta }_{g_{ϵ}}u=0$$

$${\mathfrak{m}}_{ADM}(h_{\tau })\le 2ϵ.$$

$$2ϵ\ge {\mathfrak{m}}_{ADM}(g_{ϵ})\ge {\mathfrak{m}}_H(\partial F_{ϵ},g_{ϵ}).$$

$$\begin{array}{rl}{\mathfrak{m}}_H(\partial F_{ϵ},g_{ϵ})=& \sqrt{\frac{|\partial F_{ϵ}{|}_{g_{ϵ}}}{16\pi }}\left(1-\frac{1}{16\pi }{\int }_{\partial F_{ϵ}}{H}^{2}d{\sigma }_{g_{ϵ}}\right)\\ \ge & \sqrt{\frac{|\partial F_{ϵ}{|}_{g_{ϵ}}}{16\pi }}\left(1-\frac{1}{16\pi }{\int }_{\partial F}{H}^{2}d{\sigma }_{g_{ϵ}}\right)\\ =& \sqrt{\frac{|\partial F_{ϵ}{|}_{g_{ϵ}}}{|\partial F{|}_{g_{ϵ}}}}{\mathfrak{m}}_H(\partial F,g_{ϵ})\\ \ge & \sqrt{\frac{|\partial F_{ϵ}{|}_{g_{ϵ}}}{|\partial F{|}_{g_{ϵ}}}}({\mathfrak{m}}_H(\partial E,g)-\theta -ϵ).\end{array}$$

$$\begin{array}{rl}|\partial F{|}_{g_{ϵ}}\le & \left(|\partial E{|}_g+\theta +ϵ\right)\\ \le & \left(|\partial F_{ϵ}{|}_g+\theta +ϵ\right)\\ \le & (1+ϵ)\left(|\partial F_{ϵ}{|}_{g_{ϵ}}+\theta +ϵ\right)\end{array}$$

Full text

1Research partially supported by NSFC 11671015 and 11731001

2Research partially supported by Hong Kong RGC General Research Fund #CUHK 14301517

1. Introduction

Let $(\Omega^{n},g)$ be a compact manifold with smooth boundary $\partial\Omega$. In this work, we always assume that $\Omega$ is connected and orientable. It is an interesting question to understand the relation between the geometry of $\Omega$ in terms of scalar curvature and the intrinsic and extrinsic geometry of $\partial\Omega$ in terms of the mean curvature. The question is closely related to the notion of quasi-local mass in general relativity. On other hand, given an compact manifold $(\Sigma,\gamma)$ without boundary and given a smooth function $H$ on $\Sigma$, one basic problem in Riemannian geometry is to study: *under what kind of conditions so that $\gamma$ is induced by a Riemannian metric $g$ with nonnegative scalar curvature, for example, defined on $\Omega^{n}$, and $H$ is the mean curvature of $\Sigma$ in $(\Omega^{n},g)$ with respect to the outward unit normal vector? * These two problems are closely related and there are no satisfactory answers yet.

In this kind of study, a result was proved by the authors which implies the positivity of Brown-York quasi-local mass [2, 3], denoted by $\mathfrak{m}_{BY}(\Sigma;\Omega,g)$. For its definition please see (2.1) below. More specifically, using the quasi-spherical metrics introduced by Bartnik [1], in [15] the authors proved the following:

Theorem 1.1**.**

Let $(\Omega^{3},g)$ be a compact, connected Riemannian manifold with nonnegative scalar curvature, and with compact mean-convex boundary $\partial\Omega$, which consists of spheres with positive Gaussian curvature. Then,

[TABLE]

for each component $\Sigma_{\ell}\subset\partial\Omega$, $\ell=1,\ldots,k$. Moreover, equality holds for some $\ell=1,\ldots,k$ if and only if $\partial\Omega$ has only one component and $(\Omega,g)$ is isometric to a domain in $\mathbb{R}^{3}$.

Clearly Theorem 1.1 provides a necessary condition for a boundary data $(\Sigma,\gamma,H)$ to be the one induced by a Riemannian metric defined on the ambient manifold and with nonnegative scalar curvature. Here $\gamma$ is a metric on $\Sigma$ with quasi positive Gaussian curvature. The existence of qausi-spherical metric in the proof of the theorem uses the fact that the mean curvature is positive at the boundary. Otherwise, it is unclear if one can construct such kind of metrics, see [1, 16]. With these facts in mind, it is natural to ask if Theorem 1.1 is still true in a more general context. In this note, we consider the problem in the situation of quasi-positive boundary data. Here a function defined on a set is said to be quasi positive if it is nonnegative and is positive somewhere. The specific results are the following:

Theorem 1.2**.**

Let $(\Omega,g)$ be a compact three manifold with smooth boundary $\partial\Omega$. Let $\Sigma$ be a component of $\partial\Omega$. Assume the following:

1. (a)

$\partial\Omega$* has nonnegative mean curvature.* 2. (b)

$\Sigma$* has quasi positive Gaussian curvature.* 3. (c)

$(\Omega,g)$* has nonnegative scalar curvature.*

Then we have:

1. (i)

Positivity*: $\mathfrak{m}_{BY}(\Sigma;\Omega,g)\geq 0.$* 2. (ii)

Rigidity*: Suppose $\mathfrak{m}_{BY}(\Sigma;\Omega,g)=0$, then $\partial\Omega$ is connected, $\Omega$ is homeomorphic to the unit ball in $\mathbb{R}^{3}$ and $(\Omega,g)$ is a domain in $\mathbb{R}^{3}$.*

We first remark that in case $\partial\Omega$ has quasi positive Gaussian curvature and has positive mean curvature or $\partial\Omega$ has positive Gaussian curvature and has nonnegative mean curvature, then the nonnegativity part of Theorem 1.2 was proved in [16] and [14] respectively. However, the rigidity part in the first instance was studied in [16] but not solved very satisfactorily. The rigidity part in the second instance was not addressed in [14].

To show Theorem 1.1 we used the method of quasi-spherical metric introduced by Bartnik [1]. However, if the mean curvature is only assumed to be nonnegative, a parabolic equation involved in the quasi-spherical metric may be degenerated. To overcome this difficult, in case $\partial\Omega$ is disconnected, we adopt a careful conformal perturbation on the ambient metric $g$ so that one can use Theorem 1.1 and its generalization to the case that the boundary has positive mean curvature and quasi-positive Gaussian curvature [16]. In case $\partial\Omega=\Sigma$, we use an approximation so that the mean curvature is positive but the scalar curvature may be bounded by a small negative constant. We then embed the boundary to an hyperbolic space with negative constant curvature which is small, and use a result in [18] to get nonnegativity of Brown-York mass.

We prove the rigidity part of Theorem 1.2, first we show that if the Brown-York mass is zero, then $\Omega$ is homeomorphic to the unit ball in $\mathbb{R}^{3}$ and $g$ is scalar flat. Then we need to show that $g$ is Ricci flat. By suitable approximations, as in [7] , one can construct a weak solution of the inverse mean curvature flow (IMCF) in $(\Omega,g)$ with a point $p\in\Omega$ as the initial data (see Lemma 3.3 below). We then approximate $g$ by metrics so that $\Sigma$ has positive Gaussian curvature and positive mean curvature, and so that it also has zero scalar curvature outside certain level sets of the IMCF. We can show that the level sets near $p$ have zero Hawking mass. Using the method as in the work of Husiken-Ilmanen [7], one then conclude that $g$ is Ricci flat near $p$.

It is still an open question whether the Brown-York mass is nonnegative if the mean curvature is negative somewhere.

The remaining part of the paper goes as follows: in the section 2, we prove the positivity result Theorem 1.2; in the section 3, we prove the rigidity result of the theorem.

Acknowledgment: The authors would like to thank Man-Chuen Cheng for many useful discussions.

2. Positivity

Let us first clarify the definition of Brown-York mass. Let $(\Omega,g)$ be compact three manifold with smooth boundary $\partial\Omega$. Let $\Sigma$ be a connected component of $\partial\Omega$ with induced metric $\gamma$. Suppose the Gaussian curvature of $(\Sigma,\gamma)$ is quasi positive. Then it can be $C^{1,1}$ isometrically embedded in $\mathbb{R}^{3}$ as a convex surface with mean curvature $H_{0}$ which is defined almost everywhere in $\Sigma$. Moreover,

[TABLE]

is well-defined and is positive, see [5, 6, 16]. It is well-defined in the sense that it is the same for any $C^{1,1}$ isometric embedding. Here and below mean curvature is computed with respect to the unit outward normal and the mean curvature of the boundary of the unit ball in $\mathbb{R}^{3}$ is 2. Hence one can define the Brown-York mass [2, 3] of $\Sigma$ in $(\Omega,g)$ by

[TABLE]

Here $H$ is the mean curvature of $\Sigma$ in $(\Omega,g)$. In this section, we want to prove on the positivity of Brown-York mass in Theorem 1.2.

Remark 2.1*.*

We always use the following fact. Suppose the scalar curvature $R$ of $(\Omega,g)$ is nonnegative. Let $u$ be the solution of

[TABLE]

Then $u$ is positive, so that $u^{4}g$ has zero scalar curvature and the mean curvature of $\partial\Omega$ with respect to $u^{4}g$ is no less than its mean curvature with respect to $g$.

Lemma 2.1**.**

Let $(\Omega,g)$ be a compact three manifold with smooth boundary $\partial\Omega$ and with nonnegative scalar curvature. Let $\Sigma$ be a component of $\partial\Omega$ as in Theorem 1.2. Suppose $\partial\Omega\setminus\Sigma\neq\emptyset$, then

[TABLE]

Proof.

In the following, the area element of $\partial\Omega$ with respect to the metric induced by $g$ will be denoted by $d\sigma_{g}$, and the mean curvature will be denoted by $H_{g}$, etc. Let $\gamma=g|_{T(\Sigma)}$ and let $H_{0}$ be the mean curvature when $(\Sigma,\gamma)$ is $C^{1,1}$ isometrically embedded in $\mathbb{R}^{3}$

By Remark 2.1, we may assume that the scalar curvature of $(\Omega,g)$ is zero. Moreover, since $\int_{\Sigma}H_{0}d\sigma_{g}>0$, we may assume that $H(x_{0})>0$ somewhere. Let $\Sigma^{\prime}=\partial\Omega\setminus\Sigma\neq\emptyset$.

First, we want to find a smooth metric $g_{1}$ on $\overline{\Omega}$ such that

1. (i)

$g_{1}$ has zero scalar curvature; 2. (ii)

the mean curvature $H_{g_{1}}$ of $\partial\Omega$ is positive; and 3. (iii)

$g$ and $g_{1}$ induce the same metric on $\Sigma^{\prime}$.

To construct $g_{1}$, let $U$ be a neighborhood of $x_{0}$ in $\Sigma$ such that $H_{g}\geq c_{0}>0$ in $U$. Let $0\leq\phi\leq 1$ be a smooth cutoff function with support in $U$ so that $\phi=1$ in a neighborhood of $x_{0}$. Given $\epsilon>0$ and let $u$ be the solution of

[TABLE]

For $\epsilon>0$ small enough, $u>0$ and has zero scalar curvature. Moreover,

[TABLE]

where $\nu$ is the unit outward normal. By the strong maximum principle $H_{g_{1}}>0$ outside $U$. Insider $U$, $H_{g}>0$ and so $H_{g_{1}}>0$ provided $\epsilon$ is small enough. Fix such an $\epsilon_{1}>0$. Note that the Gaussian curvature of $\Sigma$ may be negative somewhere. Hence $g_{1}=u^{4}g$ satisfies the conditions mentioned above. In particular, the mean curvature at $\Sigma^{\prime}$ with respect to $g_{1}$ is bounded below by some positive constant $a>0$.

Next, for any $\epsilon>0$ let $v$ be the harmonic function in $\Omega$ so that $v=1$ on $\Sigma$ and $v=1-\epsilon$ on $\Sigma^{\prime}$. Then for $\epsilon$ small enough, $v^{4}g$ is a smooth metric on $\overline{\Omega}$ such that the mean curvature of $\Sigma$ with respect to $v^{4}g$ is larger than the mean curvature with respect to $g$. Moreover, the mean curvature of $\Sigma^{\prime}$ with respect to $v^{4}g$ is bounded in absolute value by $\frac{a}{2}$, provided $\epsilon$ is small enough. Choose such an $\epsilon_{2}>0$. Let $g_{2}=v^{4}g$. Then $g_{2},g$ induce the same metric on $\Sigma$ and $(1-\epsilon_{2})^{4}g_{1}$ and $g_{2}$ induce the same metric on $\Sigma^{\prime}$.

Let $M_{1}=\Omega$ with metric $(1-\epsilon_{2})^{4}g_{1}$ and $M_{2}=\Omega$ with metric $g_{2}$. We can glue the $M_{1}$ and $M_{2}$ along $\Sigma^{\prime}$. Denote the resulting manifold by $M_{3}$ and the resulting metric by $g_{3}$. Then the boundary of $M_{3}$ consists of two copies of $\Sigma$ denoted by $\Sigma_{1}$ and $\Sigma_{2}$. Moreover the following are true:

1. (i)

$g_{3}$ is smooth except along $\Sigma^{\prime}$. Moreover, $g_{3}$ is Lipschitz and is smooth on each side of $\Sigma^{\prime}$. 2. (ii)

The scalar curvature of $g_{3}$ is zero away from $\Sigma^{\prime}$. 3. (iii)

The mean curvature of $\Sigma_{1}$ and $\Sigma_{2}$ are positive. 4. (iv)

The mean curvature jump at $\Sigma^{\prime}$ is positive. Namely, if we choose the unit normal pointing outside $\Sigma^{\prime}$ in $M_{1}$, then the mean curvature jump is at least $a-\frac{a}{2}=\frac{a}{2}>0$. 5. (v)

$g$ and $g_{3}$ induce the same metric on $\Sigma$ which corresponds to $\Sigma_{2}$. 6. (vi)

The mean curvature of $\Sigma_{2}$ with respect to $g_{3}$ is larger than the mean curvature of $\Sigma$ with respect to $g$.

We claim that

[TABLE]

If the claim is true, then by (v) and (vi) above, we conclude the lemma is true.

To prove the claim we further glue $M_{3}$ along $\Sigma_{1}$. Denote the resulting manifold by $M_{4}$ and the resulting metric by $g_{4}$. The boundary of $M_{4}$ consists of two copies of $\Sigma_{2}$, denoted by $\widetilde{\Sigma}_{1},\widetilde{\Sigma}_{2}$. The following are true:

1. (i)

$g_{4}$ is smooth except along those parts coming from $\Sigma^{\prime}$ or from $\Sigma_{1}$. Moreover, $g_{4}$ is Lipschitz and is smooth on each side of these surfaces. 2. (ii)

The scalar curvature of $g_{4}$ is zero away from those parts coming from $\Sigma^{\prime}$ or from $\Sigma_{1}$. 3. (iii)

The mean curvature of $\widetilde{\Sigma}_{1}$ and $\widetilde{\Sigma}_{2}$ with respect to $g_{4}$ are positive. In fact they are equal the mean curvature of $\Sigma_{2}$ with respect to $g_{3}$. 4. (iv)

The mean curvature jump at those parts coming $\Sigma^{\prime}$ or $\Sigma_{1}$ are positive, because the mean curvature of $\Sigma_{1}$ with respect to $g_{3}$ is positive. 5. (v)

$\widetilde{\Sigma}_{1},\widetilde{\Sigma}_{2}$ with respect to the induced metric from $g_{4}$ is isometric to $(\Sigma,g|_{T(\Sigma)})$.

By [9, Theorem 3.3], there exists a smooth metric $h$ on $M_{4}$ with nonnegative scalar curvature so that $h$, and $g_{4}$ induce the same metric on $\partial M_{4}$ and

[TABLE]

Moreover, $H_{h}>0$ on $\partial M_{4}$. Since each component of $\partial M_{4}$ with metric induced by $h$ is isometric to $\Sigma$ with metric induced by $g$, it has quasi positive Gaussian curvature. By [16, Theorem 0.2], we conclude that

[TABLE]

Hence the claim is true. This completes the proof of the lemma.

∎

Lemma 2.2**.**

Let $(\Omega,g)$ and $\Sigma$ be as in Theorem 1.2. Suppose $\partial\Omega=\Sigma$, then

[TABLE]

Proof.

By Remark 2.1, we may assume that $g$ is scalar flat. Note that $\partial\Omega=\Sigma$ is a sphere because its Gaussian curvature is quasi positive. Moreover, we may assume the mean curvature $H$ of $\Sigma$ is quasi positive. Let $x_{0}\in\Sigma$ with $H(x_{0})>0$. Let $U$ be an neighborhood of $x_{0}$ in $\Sigma$ such that $H_{g}\geq c_{0}>0$ in $U$. Let $0\leq\phi\leq 1$ be a smooth cutoff function with support in $U$ so that $\phi=1$ in a neighborhood of $x_{0}$. Given $\epsilon>0$ and let $u=u(\epsilon)$ be the solution of

[TABLE]

For $\epsilon>0$ small enough, $g(\epsilon)=u^{4}g$ has zero scalar curvature so that $\partial\Omega$ has positive mean curvature. Let $\gamma(\epsilon)$ be the metric on $\Sigma$ induced by $g(\epsilon)$ and let $K(\epsilon)$ be the Gaussian curvature of $\Sigma$ with respect to $\gamma(\epsilon)$. Then

[TABLE]

where $\kappa(\epsilon)>0$, $\kappa(\epsilon)\to 0$ as $\epsilon\to 0$. We can isometrically embed $(\Sigma,\gamma(\epsilon))$ in $\mathbb{H}_{-\kappa^{2}(\epsilon)}$ as a strictly convex surface in the ball model defined in the ball

[TABLE]

by [13]. Moreover, we may assume the origin is inside the embedded surface. Let $H(\epsilon)$ be the mean curvature of $\Sigma$ with respect to $g(\epsilon)$ and let $H_{\kappa(\epsilon)}$ be the mean curvature when $(\Sigma,\gamma(\epsilon))$ is isometrically embedded in the hyperbolic space $\mathbb{H}_{-\kappa^{2}(\epsilon)}$ with constant curvature $-\kappa(\epsilon)$. By [18], we have

[TABLE]

where $r$ is the distance from the origin in $\mathbb{H}_{\kappa}(\epsilon)$.

Observe that we can find $\epsilon_{i}\to 0$ such that $g(\epsilon_{i})\to g$ in $C^{\infty}$ norm on $\overline{\Omega}$. Hence the intrinsic diameter of $(\Sigma,\gamma(\epsilon_{i}))$ is bounded by a constant independent of $i$, we conclude that $r$ is bounded by a constant independent of $i$. By [8, p.7152-7154], one can choose $\epsilon_{i}\to 0$ such that:

• •

$H_{\kappa(\epsilon_{i})}$ are uniformly bounded from above. (Note that $H_{\kappa(\epsilon_{i})}>0$).

• •

If $\mathbf{X}_{i}=(x^{1},x^{2},x^{3})$ is the isometric embedding of $(\Sigma,\gamma(\epsilon_{i}))$, then the $C^{2}$ norm with respect to the fixed metric $\sigma$ are uniformly bounded.

Together with (2.4), we conclude that

[TABLE]

Moreover, $\mathbf{X}_{i}$ converge to a $C^{1,1}$ embedding of $(\Sigma,\sigma)$ in $\mathbb{R}^{3}$ as a convex surface. As in [16], one can conclude that

[TABLE]

where $H_{0}$ is the mean curvature of $\Sigma$ when $(\Sigma,\gamma)$ is isometrically $C^{1,1}$ embedded in $\mathbb{R}^{3}$. Here $\gamma=g|_{T(\Sigma)}$. From this the lemma follows.

∎

Proof of Theorem 1.2 (i) Positivity.

Let $(\Omega,g)$, $\Sigma$ be as in Theorem 1.2. Then by Lemmas 2.1 and 2.2, we have

[TABLE]

∎

3. Rigidity

In the section, we will prove the rigidity part in Theorem 1.2. First we have the following:

Lemma 3.1**.**

Let $(\Omega,g),\Sigma$ be as in Theorem 1.2 so that $\partial\Omega=\Sigma$. Suppose $\Omega$ is not homeomorphic to the unit ball in $\mathbb{R}^{3}$, then

[TABLE]

Proof.

Since the Gaussian curvature of $\Sigma$ is quasi positive, $\Sigma$ is a topological sphere. If $\Omega$ is a handle body, then it is homeomorphic to the unit ball. Suppose this is not the case, then $\Omega$ is not a handle body. By [10, Theorem 1’ and Proposition 1] there is an embedded minimal surface $S$ which is either a sphere or a minimal projective space inside $\Omega$.

Case 1: Suppose $S$ is a sphere. Since $S$ is orientable, there is a smooth unit normal vector field on $S$ and there is an embedding $F:S\times(-1,1)\to\Omega$ so that $F(\cdot\,,0)=S$ and the image of $F$ is a tabular neighborhood $N$ of $S$ in $\Omega$. Then $N\setminus S$ is a manifold with boundary which are two copies of $S$ with two components. Hence $\Omega\setminus S$ is a manifold with boundary which is a copy of $S$. Let $\widetilde{\Omega}$ be the connected component containing $\partial\Omega=\Sigma$ of this manifold. Then $(\widetilde{\Omega},g)$ has nonnegative scalar curvature so that $\partial\widetilde{\Omega}$ is disconnected, and $\mathfrak{m}_{BY}(\Sigma,\Omega,g)=\mathfrak{m}_{BY}(\Sigma,\widetilde{\Omega},g)$, which is positive by Lemma 2.1.

Case 2: Suppose $S$ is a projective space. $f:\mathbb{RP}^{2}\to\Omega$ is an embedding with $S=f(\mathbb{RP}^{2})$. We want to construct a double cover $p:\hat{\Omega}\to\Omega$ so that $p^{-1}(f(\mathbb{RP}^{2}))\cong\mathbb{S}^{2}$.

Let $V$ be the normal bundle of the embedding $f$. Note that $\mathbb{RP}^{2}$ has only two non-isomorphic real line bundles, namely the tautological line bundle and the trivial one. Since $\Omega$ is orientable, $V$ is isomorphic the tautological line bundle $((\mathbb{S}^{2}\times\mathbb{R})/\sim)\to(\mathbb{S}^{2}/\sim)\cong\mathbb{RP}^{2}$ with $(x,k)\sim(-x,-k)$ on $\mathbb{S}^{2}\times\mathbb{R}$.

By the tubular neighborhood theorem, there exists an open embedding $G:((\mathbb{S}^{2}\times\mathbb{R})/\sim)\cong V\to\Omega$ whose restriction on the zero section is equal to $f$. Let $\Omega^{\prime}=G((\mathbb{S}^{2}\times[-1,1])/\sim)$ and $\Omega^{\prime\prime}=\Omega\setminus G((\mathbb{S}^{2}\times(-1,1))/\sim)$. Then $\Omega=\Omega^{\prime}\cup\Omega^{\prime\prime}$ with $\Omega^{\prime}\cap\Omega^{\prime\prime}=\partial\Omega^{\prime}\cong\mathbb{S}^{2}$.

Let $\Omega_{+}$, $\Omega_{-}$ be two identical copies of $\Omega^{\prime\prime}$. Define $\phi:\mathbb{S}^{2}\times\{-1,1\}\to\Omega_{+}\sqcup\Omega_{-}$ by $\phi(x,1)=g([(x,1)])\in\Omega_{+}$ and $\phi(x,-1)=g([(x,-1)])\in\Omega_{-}$ for $x\in\mathbb{S}^{2}$. Let $\hat{\Omega}=\mathbb{S}^{2}\times[-1,1]\cup_{\phi}(\Omega_{+}\sqcup\Omega_{-})$. Then the obvious map $p:\hat{\Omega}\to\Omega$ has the desired properties. By the construction, we see that $(\hat{\Omega},\hat{g})$ has nonnegative scalar curvature and $\partial\hat{\Omega}$ two components, each of them has quasi-positive mean curvature with respect to outward unit norm vector and quasi-positive Gauss curvature. In fact, near each component, $(\hat{\Omega},\hat{g})$ is isometric to neighborhood of $\Sigma$ in $(\Omega,g)$. On the other hand, $2\mathfrak{m}_{BY}(\Sigma,\Omega,g)=\mathfrak{m}_{BY}(\partial\hat{\Omega},\hat{\Omega},g)$, which is positive by Lemma 2.1. This completes the proof of the lemma. ∎

Let $(\Omega,g)$ and $\Sigma$ be as in Theorem 1.2. Suppose $\mathfrak{m}_{BY}(\Sigma;\Omega,g)=0$. Then by Lemmas 2.1 and 3.1, we conclude that $\partial\Omega=\Sigma$ and $\Omega$ is homeomorphic to the unit ball. By Remark 2.1, we conclude that $g$ is scalar flat. Moreover, since $\Sigma$ has quasi positive Gaussian curvature, we conclude that $\Sigma$ has quasi positive mean curvature. In the rest of this section, we always assume the above facts. In remains to prove that $g$ is Ricci flat.

We need the following two lemmas.

Lemma 3.2**.**

Let $(\Omega,g)$ and $\Sigma$ be as above. For any $p$ in $\Omega$ and for any $\rho>0$ small enough, there is a sequence of smooth metrics $g_{i}$ on $\overline{\Omega}$ with the following properties:

1. (i)

$g_{i}\to g$* in $C^{\infty}$ norm in $\overline{\Omega}$.* 2. (ii)

$\Sigma$* has positive mean curvature $H_{i}$ with respect to $g_{i}$.* 3. (iii)

Let $\gamma_{i}$ be the induced metric of $g_{i}$ on $\Sigma$. Then the Gaussian curvature of $(\Sigma,\gamma_{i})$ has positive Gaussian curvature. 4. (iv)

The scalar curvature of $g_{i}$ is zero outside $B(p,2\rho)$. 5. (v)

The mean curvature of $\partial B_{g}(p,s)$ with respect to $g_{i}$ is positive for all $s<2\rho$ for all $i$. 6. (vi)

$\mathfrak{m}_{BY}(\Sigma;\Omega,g_{i})\to 0$* as $i\to\infty$.*

Proof.

Let $\rho>0$ be small enough so that $\partial B_{g}(p,s)$ is diffeomorphic to the sphere so that its mean curvature is larger than $1/s$ for all $0<s<2\rho$. Fix a smooth cutoff function $\phi\geq 0$ so that $\phi=1$ in $B(p,\rho)$ and $\phi=0$ outside $B(p,2\rho)$. Let $v$ be the solution of $\Delta_{g}v=\epsilon\phi$ in $\Omega$ and $v=1$ on $\Sigma$. Then for $\epsilon>0$ small enough, $v>0$. Let $g_{\epsilon}=v^{4}g$. For $\epsilon$ small enough, $g_{\epsilon}$ satisfies:

• •

$g_{\epsilon}\to g$ in $C^{\infty}$ norm in $\overline{\Omega}$.

• •

The scalar curvature of $g_{\epsilon}$ is zero outside $B(p,2\rho)$.

• •

The mean curvature of $\Sigma$ with respect to $g_{\epsilon}$ is positive. This follows by strong maximum principle that $\frac{\partial v}{\partial\nu}>0$ where $\nu$ is the unit outward normal of $\Sigma$ with respect to $g$.

Since $v=1$ on $\Sigma$, the metrics induced by $g,g_{\epsilon}$ are equal, and will be denoted by $\gamma$. In particular, the Gaussian curvature of $\Sigma$ does not change. If the Gaussian curvature of $(\Sigma,\gamma)$ is positive, then $g_{\epsilon}$ are the required metrics. Otherwise, we can find a smooth function $\eta$ on $\Sigma$ such that $\eta\leq 0$, $\Delta_{\gamma}\eta=-1$ in an open set containing $\{K=0\}$. For fixed $\epsilon>0$, for $\tau>0$, and let $w$ be the solution of $\Delta_{g_{\epsilon}}w=0$ in $\Omega$ so that $w=\mathrm{exp}(\frac{1}{2}\tau\eta)$. Let $h_{\tau}=w^{4}g_{\epsilon}$. Then

• •

$h_{\tau}\to g_{\epsilon}$ in $C^{\infty}$ norm in $\overline{\Omega}$ as $\tau\rightarrow 0$.

• •

The scalar curvature of $h_{\tau}$ is zero outside $B(p,2\rho)$.

• •

The mean curvature of $\Sigma$ is positive, provided $\tau$ is small enough.

• •

The Gaussian curvature of $\Sigma$ with respect to the metric induced by $h_{\tau}$ is positive provided $\tau$ is small enough.

From these, it is easy to see the lemma is true.

∎

The following lemma is basically from [7].

Lemma 3.3**.**

Let $(\Omega,g)$, $\Sigma$ be as above. For any $p\in\Omega$, there is a weak solution for the inverse mean curvature flow in $(\Omega,g)$ with $p$ as the initial data.

Proof.

Let $U$ be a small neighborhood of $\partial\Omega$, then extend $\Omega\cup U$ to be Euclidean near infinity, the resulting metric is denoted by $\hat{g}$.

Let us consider the inverse mean curvature flow (IMCF) in $(M,\hat{g})$ with $\partial B_{r}(p)$ as the initial data where $r>0$ is small enough. By Theorem 3.1 in [7], there is a weak solution $u_{r}$ to this IMCF with $u_{r}|_{\partial B_{r}(p)}=0$ and

[TABLE]

for any $0<\rho\leq\sigma(x)$, here $C$ is a universal constant independent on $\rho$ and $r$, $\sigma(x)$ is defined in Definition 3.3 in [7], i.e. for any $x\in\Omega$, let $\tau(x)\in(0,\infty]$ be the supremum of radii $r$ such that $B_{r}(x)\subset\Omega$, and

[TABLE]

and there is a $C^{2}$ function $p$ on $B_{r}(x)$ such that $p(x)=0$, $p\geq d^{2}(,x)$, and $|\nabla p|\leq 3d(,x)$, $\nabla^{2}p\leq 3g$ on $B_{r}(x)$, define $\sigma(x)=\min\{\tau(x),d(x,\partial\Omega)\}$. Let $\Omega^{\prime}\subset\subset\Omega$ with $dist(\partial\Omega^{\prime},\partial\Omega)$ being any fixed small number and $p\in\Omega^{\prime}$. Without loss of the generality, it suffices to consider the case that $x\in\Omega^{\prime}$, so, we may assume $\sigma(x)\geq\sigma_{0}$ for any $x\in\Omega^{\prime}$, here $\sigma_{0}$ is a fixed number depends only on $dist(\partial\Omega^{\prime},\partial\Omega)$ and $(\Omega,g)$.

Let us choose $r$ small enough so that $\sup_{\partial B_{r}(p)}H_{+}\leq\frac{3}{r}$. Now, we claim that for any $x\in\Omega^{\prime}$

[TABLE]

here $C$ is a universal constant independent on $r$, $d(x,p)$ is the distance function to $p$ with respect to the metric $g$.

In fact, if $d(x,p)\leq 4r$, then we take $\rho=\frac{r}{2}$, here we assume $r\leq\frac{\sigma_{0}}{2}$, we get (3.1); if $d(x,p)>4r$, let $\rho=\min\{\frac{1}{2}dist(x,p),\frac{\sigma_{0}}{2}\}$, together with the fact $dist(x,p)\leq\Lambda\sigma_{0}$, where $\Lambda$ is a universal constant, we still get (3.1).

On the other hand, together with Theorem 2.1 in [7] and the remarks following it, we know that by taking a subsequence of $\{u_{r}\}$, denoted by $\{u_{r_{i}}\}$, there is a constant $C_{i}$ so that $\{u_{r_{i}}-C_{i}\}$ converges to the weak solution of IMCF $-\infty<u$ in $(\Omega^{\prime},g)$ with $p$ as the initial data. Note that the mean curvature of $\partial B_{r}(p)$ is positive for all $r\leq\delta$, we see that the level set of $u$ in $B_{\delta}(p)\subset\subset\Omega^{\prime}$ cannot jump, and

[TABLE]

and $-\infty<u\leq t_{0}$, here $t_{0}$ is a universal constant. ∎

Let us first recall the definition of minimizing hull in $\Omega$. A subset $E$ of $\Omega$ with locally finite perimeter said to be a minimizing hull in $\Omega$ if $|\partial^{*}E\cap K|\leq|\partial^{*}F\cap K|$ for any set $F\subset\Omega$ with locally finite perimeter such that $F\supset E$ and $F\setminus E\Subset\Omega$ and for any compact set $K$ with $F\setminus E\subset K\subset\Omega$. Here $\partial^{*}E,\partial^{*}F$ are the reduced boundaries of E and F respectively.

By the proof in [17, Theorem 2.5], we see that for $t$ small enough, the slice $N_{t}=\partial\{u<t\}$ of the weak IMCF in Lemma 3.3 is the boundary of a minimizing hull in $(\Omega,g)$ with $C^{1,\alpha}$ smooth and $\int_{N_{t}}|A|^{2}d\sigma<\infty$, and $\mathfrak{m}_{H}(N_{t})\geq 0$.

We are ready to prove the rigidity part of Theorem 1.2.

Proof of Theorem 1.2 (ii) Rigidity.

Let $p\in\Omega$. Suppose $g$ is not flat near $p$. Choose $r>0$ be small enough with $B(p,2r)\Subset\Omega$, so that $\partial B(p,s)$ is a sphere with mean curvature at least $1/s$ for all $s<2r$. Then by Lemma 3.3 and [7], one can find a solution to the IMF given by a locally Lipschitz function $u$, so that for some $a$, the following are true: (i) $E_{t}=\{u<t\}$ is precompact in $B(x,r)$ for $t<a$; (ii) $\partial E_{t}$ is connected; (iii) $E_{t}$ is a minimizing hull in $(\Omega,g)$; (iv) $\mathfrak{m}_{H}(\partial E_{t},g)>0$, for $t<a$.

Fix $t_{0}<a$ so that $\mathfrak{m}_{H}(\partial E_{t_{0}},g)\geq b$ for some $b>0$. In the following we denote $E_{t_{0}}$ by $E$. For any $\theta>0$ small enough, we can find $E\subset F\Subset B(x,r)$ such that

[TABLE]

Moreover $\partial F$ is smooth. Note that $F$ depends on $\theta$.

Since $p\in E_{t_{0}}$ which is open, we can find $r>\rho>0$ such that $B(p,2\rho)\Subset E$.

Next, we want to approximate $g$. By the Lemma 3.2 , for any $\epsilon>0$ small enough, we can find a smooth metric $g_{\epsilon}$ on $\overline{\Omega}$ so that (i) $||g-g_{\epsilon}||_{C^{4}}\leq\epsilon$; (ii) $\Sigma$ has positive mean curvature $H_{\epsilon}$ with respect to $g_{\epsilon}$; (iii) The Gaussian curvature of $(\Sigma,g_{\epsilon}|_{T(\Sigma)})$ has positive Gaussian curvature. (iv) the scalar curvature of $g_{\epsilon}$ is zero outside $B(p,2\rho)$; (v) The mean curvature of $\partial B(p,s)$ with respect to $g_{\epsilon}$ is positive for all $s<2r$; (vi) $\mathfrak{m}_{BY}(\Sigma,\Omega,g_{\epsilon})\leq\epsilon$; (vii) $|\partial F|_{g_{\epsilon}}\leq|\partial E|_{g}+\theta+\epsilon$, $\mathfrak{m}_{H}(\partial F,g_{\epsilon})\geq\mathfrak{m}_{H}(\partial E,g))-\theta-\epsilon>0$.

By (ii), (iii), we can glue $\Omega$ to the exterior of the a convex set in $\mathbb{R}^{3}$ so that the scalar curvature outside the convex set is zero and is asymptotically flat. Denote the manifold by $M$. We still denote this metric as $g_{\epsilon}$. Note that $g_{\epsilon}$ has zero scalar curvature outside $B(x,2r)$. However, $g_{\epsilon}$ may have negative scalar curvature inside $B(p,2\rho)$. By the monotonicity in qausi-spherical metric [15], using the Lemma 3.2 (vi) we may choose $g_{\epsilon}$ so that

[TABLE]

Fix such an $\epsilon$. Using the method of Miao [11], for $\tau>0$ small enough, we can find metrics $h_{\tau}$ so that $h_{\tau}=g_{\epsilon}$ outside $\{x\in M|d_{g_{\epsilon}}(x,\Sigma)<\tau\}$ and the scalar curvature inside $\{x\in M|d_{g_{\epsilon}}(x,\Sigma)<\tau\}$ is uniformly bounded. Let $R_{\tau}$ be the scalar curvature of $g_{\epsilon}$. One can find a positive solution of

[TABLE]

with $u\to 1$ near infinity. Here $\widetilde{R}_{\tau}=R_{\tau}$ in $\{x\in M|d_{g_{\epsilon}}(x,\Sigma)<\tau\}$ and $\widetilde{R}_{\tau}=0$ outside this set. Note that $\widetilde{R}_{\tau}$ is smooth. Hence one can approximate $g_{\epsilon}$ by a smooth metrics $h_{\tau}=u^{4}g_{\epsilon}$ on the manifold so that, $h_{\tau}$ has zero scalar curvature outside $B(p,2\rho)$ and

[TABLE]

Moreover, $h_{\tau}\to g_{\epsilon}$ uniformly in $M$, $h_{\tau}\to g_{\epsilon}$ in $C^{\infty}$ norm in any compact set away from $\Sigma$.

Note that the mean curvature of $\Sigma$ with respect to $g_{\epsilon}$ is positive and $\mathfrak{m}_{H}(\partial F,g_{\epsilon})>0$, one can find $F_{\epsilon}$ which is the minimizing hull of $F$ with respect to $g_{\epsilon}$ inside $\Omega$. $F_{\epsilon}$ exists because the mean curvature of $\Sigma=\partial\Omega$ is positive with respect to $g_{\epsilon}$. Then $F_{\epsilon}\Subset\Omega$ and is connected because $M$ is homeomorphic to $\mathbb{R}^{3}$. Using the fact that the scalar curvature of $g_{\epsilon}$ is zero outside $\partial F$, one can proceed as in the proof [16, Theorem 3.1], to obtain

[TABLE]

On the other hand, the mean curvature of $\partial F_{\epsilon}$ is zero on $\partial F_{\epsilon}\setminus\partial F$ is equal to the mean curvature of $\partial F$ on $\partial F_{\epsilon}\cap\partial F$

[TABLE]

Now

[TABLE]

and

[TABLE]

here we may assume that $(1+\epsilon)^{-1}g\leq g_{\epsilon}\leq(1+\epsilon)g$. Hence

[TABLE]

Since $(\mathfrak{m}_{H}(\partial E,g)-\theta-\epsilon)>0$ provided $\theta,\epsilon$ are small enough, we have

[TABLE]

Let $\epsilon\to 0$ and then let $\theta\to 0$, we have

[TABLE]

This is a contradiction.

∎

Remark 3.1*.*

It is not difficult to see that by the arguments in the above proof of rigidity, we may also get $\mathfrak{m}_{BY}(\Sigma;\Omega,g)\geq 0$ in case $\Omega$ is homeomorphic to a ball.