Symmetry properties of positive solutions for fully nonlinear elliptic systems

TL;DR

This paper studies symmetry properties of positive solutions for fully nonlinear elliptic systems, extending classical methods to nonsmooth cases and applying results to uniqueness and symmetry in specific systems.

Contribution

It develops a unified moving planes approach for nonsmooth nonlinearities and applies it to establish symmetry and uniqueness results in elliptic systems.

Findings

Symmetry of solutions in bounded domains with Dirichlet boundary conditions.

Extension of moving planes method to nonsmooth nonlinearities.

Uniqueness of positive solutions for certain Lane-Emden systems.

Abstract

We investigate symmetry properties of positive solutions for fully nonlinear uniformly elliptic systems, such as $$ F_i \,(x,Du_i,D^2u_i) +f_i \,(x,u_1, \ldots , u_n,Du_i)=0, \;\; 1 \leq i \leq n, $$ in a bounded domain $\Omega$ in $\mathbb{R}^N$ with Dirichlet boundary condition $u_1=\ldots,u_n=0$ on $\partial\Omega$. Here, $f_i $'s are nonincreasing with the radius $r=|x|$, and satisfy a cooperativity assumption. In addition, each $f_i $ is the sum of a locally Lipschitz with a nondecreasing function in the variable $u_i$, and may have superlinear gradient growth. We show that symmetry occurs for systems with nondifferentiable $f_i$'s by developing a unified treatment of the classical moving planes method in the spirit of Gidas-Ni-Nirenberg. We also present different applications of our results, including uniqueness of positive solutions for Lane-Emden systems in the subcritical case in a ball, and symmetry for a class of systems with natural growth in the gradient.

Equations

$$F_i(x,D{u}_i,D^2{u}_i)+f_i(x,u_1,\dots ,u_n,D{u}_i)=0,1\le i\le n,$$

$$\displaystyle\left\{\begin{array}[]{rclcl}F_{i}\,(x,Du_{i},D^{2}u_{i})+f_{i}\,(x,u_{1},\ldots,u_{n},Du_{i})&=&0&\mbox{in}&\;\Omega\\ u_{i}&>&0&\mbox{in}&\;\Omega\\ u_{i}&=&0&\mbox{on}&\partial\Omega,\;\;\,1\leq i\leq n,\end{array}\right.$$

$$(H_0),(H_1),(H_2),(H_3),(H_4),\text{ and }(H_5).$$

$$\displaystyle\left\{\begin{array}[]{rclcc}\Delta u+v^{q}&=&0&\mbox{in}&\;B\\ \Delta v+u^{p}&=&0&\mbox{in}&\;B\\ u,\,v&>&0&\mbox{in}&\;B\\ u,\,v&=&0&\mbox{on}&\partial B\end{array}\right.$$

$$\frac{N}{p+1}+\frac{N}{q+1}>N-2,p,q>0,N\ge 1,$$

$${\mathcal{M}}_i^{+}(X):=\underset{{\alpha }_{i}I\le A\le {\beta }_{i}I}{\sup }\mathrm{tr}(AX),{\mathcal{M}}_i^{-}(X):=\underset{{\alpha }_{i}I\le A\le {\beta }_{i}I}{\inf }\mathrm{tr}(AX)$$

$${\mathcal{L}}^{\pm }[w]:={\mathcal{M}}_1^{\pm }(D^{2}w)\pm \gamma |Dw|,{\tilde{\mathcal{L}}}^{\pm }[w]:={\mathcal{M}}_1^{\pm }(D^{2}w)\pm (\gamma +\mu )|Dw|,$$

$$\text{if }(x_1\dots ,x_N)\in \Omega ,\text{ then }(t,x_2,\dots ,x_N)\in \Omega ,\text{ for all }t\in [-x_1,x_1].$$

$$T_{\lambda }=\Omega \cap \{x_1=\lambda \}.$$

$${\Sigma }_{\lambda }=\Omega \cap \{x_1>\lambda \},$$

$${\Lambda }_2=\inf \{\lambda <R;{\Sigma }_{\mu }^{\mu }\subset \Omega ,\text{ for all }\mu \in (\lambda ,R)\}.$$

$${\Lambda }_1=\inf \{\lambda <R;{\Sigma }_{\mu }\text{ does not reach positions (I) and (II), for all }\mu \in (\lambda ,R)\}.$$

$$U_i^{\lambda }=u_i^{\lambda }-u_i,\text{ for }{u}_i^{\lambda }(x)=u_i(x^{\lambda }),$$

$$-F_i(x,D{u}_i^{\lambda }(x),D^2{u}_i^{\lambda }(x))\ge f_i(x,u_1^{\lambda }(x),\dots ,u_n^{\lambda }(x),D{u}_i^{\lambda }(x))\text{ in }{\Sigma }_{\lambda }.$$

$$({\tilde{H}}_0),(H_1),(H_2),(H_4),(H_5),\text{ and }(H_7).$$

$$c_{ij}(x,h,p)=\frac{1}{h}\{f_i(x,{\text{w}}^j,p)-f_i(x,\text{u},p)\}\text{ if }h\ne 0,\text{ and }{c}_{ij}(x,0,p)=0,$$

$$\displaystyle\left\{\begin{array}[]{rclcc}F(x,Du,D^{2}u)+f(x,u,v,Du)&=&0&\mbox{in}&\;\Omega\\ G(x,Du,D^{2}u)+g(x,u,v,Dv)&=&0&\mbox{in}&\;\Omega\\ u,\,v\in C^{2}(\overline{\Omega}),\,\;\;u,\,v&>&0&\mbox{in}&\;\Omega\\ u,\,v&=&0&\mbox{on}&\partial\Omega.\end{array}\right.$$

$$-{\mathcal{L}}^{-}[U^{\lambda }]$$

$$=\{f(x,u^{\lambda },v^{\lambda },D{u}^{\lambda })-f(x,u^{\lambda },v,D{u}^{\lambda })\}+\{f_1(x,u^{\lambda },v,D{u}^{\lambda })-f_1(x,u,v,D{u}^{\lambda })\}$$

$$+\{f_2(x,u^{\lambda },v,D{u}^{\lambda })-f_2(x,u,v,D{u}^{\lambda })\}+\{f(x,u,v,D{u}^{\lambda })-f(x,u,v,Du)\}$$

$$\ge c_{12}(x,V^{\lambda },D{u}^{\lambda })V^{\lambda }-d_f{U}^{\lambda }-\mu |D{U}^{\lambda }|\text{ in }{\Sigma }_{\lambda },$$

$$u_{x_1}<0\text{ on }\Omega \cap B_{\epsilon }(x_0).$$

$$-{\mathcal{L}}^{-}[u]$$

$$=\{f(x,u,v,Du)-f(x,u,0,Du)\}+\{f_1(x,u,0,Du)-f_1(x,0,0,Du)\}$$

$$+\{f_2(x,u,0,Du)-f_2(x,0,0,Du)\}+\{f(x,0,0,Du)-f(x,0,0,0)\}$$

$$\ge c_{12}(x,v,Du)v-d_{f}u-\mu |Du|.$$

$$Du(x)\cdot t=0\text{ for all }x\in \partial \Omega \text{ and }t\in T_x.$$

$$t_1\cdot (D^{2}u(x_0)\cdot t_0)=0.$$

$$0\le -{\mathcal{M}}_1^{+}(D^{2}u(x_0))\le -F(x_0,0,D^{2}u(x_0))=f(x_0,0,0,0).$$

$$f(\tilde{z},0,0,0)\le f(\tilde{x},0,0,0)<0.$$

$$\Lambda =\inf \{\lambda >0;U^{\mu }>0,V^{\mu }>0\text{ in }{\Sigma }_{\mu },\text{ for all }\mu \in (\lambda ,R)\}.$$

$$u_{x_1},v_{x_1}<0\text{ on }{T}_{\Lambda -\epsilon }.$$

$$0\ge U^{{\lambda }_k}(x_k)=u(x_k^{{\lambda }_k})-u(x_k)=2u_{x_1}(y_k)({\lambda }_k-x_{k,1}).$$

Full text

Symmetry properties of positive solutions for fully nonlinear elliptic systems

Ederson Moreira dos Santos111ederson@icmc.usp.br.

Instituto de Ciências Matemáticas e de Computação, Universidade de São Paulo, Brazil

Gabrielle Nornberg222gabrielle@icmc.usp.br.

Instituto de Ciências Matemáticas e de Computação, Universidade de São Paulo, Brazil

Abstract. We investigate symmetry properties of positive solutions for fully nonlinear uniformly elliptic systems, such as

[TABLE]

in a bounded domain $\Omega$ in $\mathbb{R}^{N}$ with Dirichlet boundary condition $u_{1}=\ldots,u_{n}=0$ on $\partial\Omega$. Here, $f_{i}$’s are nonincreasing with the radius $r=|x|$, and satisfy a cooperativity assumption. In addition, each $f_{i}$ is the sum of a locally Lipschitz with a nondecreasing function in the variable $u_{i}$, and may have superlinear gradient growth. We show that symmetry occurs for systems with nondifferentiable $f_{i}$’s by developing a unified treatment of the classical moving planes method in the spirit of Gidas-Ni-Nirenberg. We also present different applications of our results, including uniqueness of positive solutions for Lane-Emden systems in the subcritical case in a ball, and symmetry for a class of systems with natural growth in the gradient.

Keywords. Positive solution; Elliptic system of fully nonlinear equations; Symmetry; Moving planes.

MSC2010. 35J47, 35J60, 35N25, 35B06, 35B50.

1 Introduction and main results

In this paper we study radial symmetry of solutions for fully nonlinear uniformly elliptic systems in the following form

[TABLE]

where $\Omega$ is a bounded $C^{2}$ domain in ${\mathbb{R}^{N}}$, $N\geq 1$, $n\geq 1$, and $f_{i}$ is not necessarily locally Lipschitz.

Symmetry properties of partial differential equations, for general, are of independent interest since it always makes sense to ask whether or not solutions inherit the same symmetry from the differential operator and from the domain of definition. On the other hand, special attention has been devoted to radial solutions of specific problems. For instance, the Lane-Emden conjecture has been fully solved in the radial setting [25, Theorem 3.1], and the uniqueness of positive radial solutions for Lane-Emden systems was proved in [12, Theorem 1.1(i)]. In particular, in these specific cases, it should be natural to reach the full result via symmetry. We adopt this procedure to prove the uniqueness of positive solution to Lane-Emden systems; cf. Corollary 1.4 ahead.

Radial symmetry has been extensively studied in the literature since the seminal works of Serrin [33] and Gidas-Ni-Nirenberg [20], which are based upon the moving planes method. The method was revisited in the influential paper [2] of Berestycki-Nirenberg, in which a central tool to start moving the planes is the maximum principle in small domains. This permitted the authors to remove the original twice differentiability assumption up to the boundary on solutions (and the regularity of the boundary), although they imposed a locally Lipschitz condition which must be true even when $u=0$. In the recent years, a lot of variants were considered. For example, assuming differentiability on the $f_{i}$’s, in [13] the authors considered a different type of symmetry, namely foliated Schwarz, related to solutions having low Morse index. On the non-Lipschitz scenario, it was performed in [17] a local moving plane method followed by a unique continuation principle.

The generalization of the pioneering method for systems was first considered in [37]. The respective version of [2] for cooperative systems in the differentiable case can be found in [15], and in the most works that have picked out this approach since then. It is not our intention here to give a full literature review. Instead we quote other few papers as [3, 11, 14, 22, 32], and references therein, in which a more clear scenario can be built related to both equations and systems.

On the other hand, in contrast to the scalar case, several prototype problems involving systems in the superlinear setting are naturally not differentiable or even not Lipschitz; e.g. (1.9) and (4.8). Up to our knowledge, symmetry results for systems without differentiability hypotheses on $f_{i}$ are not properly available. The results in this paper include these cases and provide a better understanding about symmetry properties for systems by featuring the essence of Gidas-Ni-Nirenberg technique. Consequently, we clarify some divergences that appeared in the literature over the past years since the work [34]; see (3) where the cooperativity assumption ($H_{4}$) is used. We focus on relaxing Lipschitz or differentiability hypotheses on $f_{i}$, even under the price of asking natural regularity on the solutions $u_{i}$ and on the domain.

Next we list our hypotheses. First and foremost, a consistent requirement over equations when dealing with radial symmetry is their rotational invariance property. For our operators $F_{i}$, this will be expressed in terms of an exclusive dependence on the eigenvalues of $D^{2}u$ and on the lengths of $Du$ and $x$. We assume, as in [35], that $\mathcal{F}_{i}:\overline{\Omega}\times\mathbb{R}^{n}\times{\mathbb{R}^{N}}\times\mathbb{S}^{N}(\mathbb{R})\rightarrow\mathbb{R}$, where $\mathcal{F}_{i}(x,\textbf{u},p,X)=F_{i}(x,p,X)+f_{i}(x,\textbf{u},p)$, with $\textbf{u}=(u_{1},\ldots,u_{n})$, $1\leq i\leq n$, satisfy the following invariance

• ($H_{0}$)

$\mathcal{F}_{i}\,(x,\textbf{u},Qp,Q^{t}XQ)=\mathcal{F}_{i}\,(x,\textbf{u},p,X)$ for all $N\times N$ orthogonal matrix $Q$, and for all $x,\textbf{u},p,X$.

Here $\mathbb{S}^{N}(\mathbb{R})$ is the subspace of symmetric matrices of order $N$ with real entrances. Concerning structure, to apply the moving planes method, as in [11], we consider on each $F_{i}$ the following condition,

• ($H_{1}$)

$\mathcal{M}_{i}^{-}(X-Y)-\gamma|p-q|\leq F_{i}\,(x,p,X)-F_{i}\,(x,q,Y)\leq\mathcal{M}^{+}_{i}(X-Y)+\gamma|p-q|$, for all $X,Y\in\mathbb{S}^{N}(\mathbb{R})$, $p,q\in{\mathbb{R}^{N}}$, and $x\in\Omega$, with $F_{i}(\cdot,0,0)\equiv 0$,

where $\mathcal{M}_{i}^{\pm}$ are the Pucci’s extremal operators; see Section 2. In addition, in order to treat possible superlinear growth in the gradient, we assume that for any $\sigma>0$, there exists $\mu_{\sigma}\geq 0$ such that

• ($H_{2}$)

$|f_{i}\,(x,\textbf{u},p)-f_{i}\,(x,\textbf{u},q)|\leq\mu_{\sigma}|p-q|\,$ for $p,q\in\overline{B}_{\sigma}$, $x\in\Omega$ and $\textbf{u}=(u_{1},\ldots,u_{n})\in\overline{B}_{\sigma}$.

For instance, we refer to [21] where a hypothesis of this nature appears to treat equations with quadratic gradient growth. Moreover, for each $i\in\{1,\ldots,n\}$ we assume

• ($H_{3}$)

$F_{i}$ and $f_{i}$ are radially symmetric, and $F_{i}+f_{i}$ is nonincreasing with $r=|x|$, for each fixed $\textbf{u},p,X$;

• ($H_{4}$)

$f_{i}=f_{i,1}+f_{i,2}$, where $f_{i,1}$ is uniformly locally Lipschitz in the component $u_{i}$, and $f_{i,2}$ is nondecreasing in $u_{i}$, whenever the remaining components $u_{j}$, for $j\neq i$, and $x,p$ are fixed;

• ($H_{5}$)

$u_{j}\mapsto f_{i}\,(x,u_{1},\ldots,u_{n},p)$ is nondecreasing if $i\neq j$, and $x$, $p$, and $u_{l}$, for $l\neq j$, are fixed.

The latter represents cooperativity for the system (1.4), which ensures the validity of the strong maximum principle; compare with [37, eq. (1.3)] in the differentiable setting, and [16, eq. (1.2)] in the linear one. Besides sufficient, this condition is also necessary in order to preserve the symmetry; see Section 4.

We use the shorthand notation ($H$) for the preceding hypotheses, that is,

[TABLE]

In what follows, we say solutions to mean the classical solutions of the Dirichlet problem (1.4), being twice differentiable up to the boundary. We stress that this is always the case if, for instance, each $f_{i}(x,\textbf{u}(x),Du_{i}(x))$ is $C^{\alpha}(\overline{\Omega})$ as a function of $x$, and $F_{i}$ is concave or convex in the $X$ entry (in particular the Pucci’s operators); see [7] and [38]. On the other hand, viscosity solutions are understood in the $C$-viscosity sense; see the next section.

We state our main results in the sequel.

Theorem 1.1**.**

Let $\Omega$ be a ball of radius $R$ centered at the origin. Assume ($H$) for all $1\leq i\leq n$. Let $(u_{1},\ldots,u_{n})$ be a solution of (1.4), where $u_{i}$’s are $C^{2}(\overline{\Omega})$ functions. Then $u_{i}$ is radially symmetric and $\partial_{r}u_{i}<0$, for all $r=|x|\in(0,R)$, $1\leq i\leq n$.

If the operators $F_{i}$, $1\leq i\leq n$, are continuously differentiable in the entry $X\in\mathbb{S}^{N}(\mathbb{R})$, it is derived the following result concerning overdetermined problems in general smooth domains for our type of systems. Consistently with the classical notation [33, 35], the involved functions do not depend on $x$.

Theorem 1.2**.**

Let $\Omega$ be a bounded $C^{2}$ domain. Assume that $F_{i}\,(p,X)$ is continuously differentiable in $X$, and that $f_{i}$ and $F_{i}$ verify ($H$) without dependence on $x$, for all $1\leq i\leq n$. Let $(u_{1},\ldots,u_{n})$ be a solution of (1.4), where $u_{i}$’s are $C^{2}(\overline{\Omega})$ functions. Assume that $\partial_{\nu}u_{i}=c_{i}$ on $\partial\Omega$, where $c_{i}$ is a constant and $\nu$ is the unit interior normal to $\partial\Omega$. Then $\Omega$ must be a ball, $u_{i}$ is radially symmetric and $\partial_{r}u_{i}<0$, for all $1\leq i\leq n$.

As already observed in [35], the $C^{1}$ hypothesis on $F_{i}$ is necessary in order to apply Serrin’s lemma [33, Lemmas 1 and 2], which is a version of Hopf lemma in domains with corners.

A model case for Theorem 1.1 consists of Pucci’s operators, or more generally $\mathcal{F}_{i}(x,\textbf{u},p,X)=\mathcal{M}_{i}^{\pm}(X)\pm\gamma_{i}(|x|)|p|\pm\mu_{i}(|x|)|p|^{2}+f_{i}(|x|,\textbf{u})$ for some bounded functions $\gamma_{i},\mu_{i}\geq 0$, and we emphasize that our results are new even for systems involving the Laplacian operator. For Theorem 1.2, a simple example is $\mathcal{F}_{i}(\textbf{u},p,X)=\mathrm{tr}(A^{i}X)+b_{i}|p|+\mu_{i}|p|^{2}+f_{i}(\textbf{u})$, where $A^{i}$ is a positive matrix, and $b_{i},\mu_{i}$ are nonnegative constants.

Regarding viscosity solutions, it is possible to obtain a restatement of Theorem 1.1 for the ones which are continuously differentiable up to the boundary. In this case we need to impose continuity of the operators $F_{i}$ in the variable $x$ as in [11, (H1)], namely

• ($H_{6}$)

$|F_{i}\,(x,p,X)-F_{i}\,(y,q,Y)|\leq\gamma\{|p-q|+\|X-Y\|+|x-y|(\|X\|+\|Y\|)\}+\omega(|x-y|(1+|p|+|q|))$

for all $X,Y\in\mathbb{S}^{N}(\mathbb{R})$, $p,q\in{\mathbb{R}^{N}}$, and $x,y\in\Omega$, for some continuous function $\omega$ with $\omega(0)=0$.

Theorem 1.3**.**

Let $\Omega$ be a ball of radius $R$ centered at the origin. Assume ($H$), ($H_{6}$), and moreover $f_{i}(\cdot,0,\ldots,0)\geq 0$ for all $1\leq i\leq n$. Let $(u_{1},\ldots,u_{n})$ be a viscosity solution of (1.4), where $u_{i}$’s are $C^{1}(\overline{\Omega})$ functions. Then $u_{i}$ is radially symmetric and $\partial_{r}u_{i}<0$, for all $r=|x|\in(0,R)$, $1\leq i\leq n$.

We stress that viscosity solutions of fully nonlinear equations usually have $C^{1,\alpha}$ regularity if the operator is continuous with respect to $x$ and has structure as in $(H_{1})$. In the presence of a superlinear and at most quadratic gradient growth this follows from [27]. We prefer to keep hypothesis $(H_{2})$, which is more general, once symmetry in itself does not require some particular growth. On the other hand, the hypothesis on the sign of $f_{i}$ is quite natural when considering moving planes throught fully nonlinear structures, see [30, 35].

As an application, we obtain uniqueness results for the following Lane-Emden system posed in a ball,

[TABLE]

where $p,q>0$, $pq\neq 1$. Namely, $pq=1$ is a separate case related to eigenvalue problems, in which the scaling $(t^{q}u,tv)$, $t\in(0,\infty)$, produces multiple eigenfunctions; see [12, 26] for instance.

It is well known that criticality plays a meaningful role in existence and nonexistence results. As far as problem (1.9) is concerned, we will be interested in the subcritical case

[TABLE]

since no solution exists in the complementary setting; see for example [24, Proposition 3.1]. Several papers prove existence of positive solution in the subcritical case for a general bounded regular domain; see for instance [6, Theorems 1.3 and 1.4] and [4] for a rather complete overview on the subject. Uniqueness is known when $pq<1$ for general bounded smooth domains; see [26, Theorem 4.1] or [5, Theorem 7.1]. However, we cannot expect a uniqueness result holding true for general domains if $pq>1$, since multiplicity has already been proved for an annulus in [23, Theorem 1.1]. In the case of a ball, uniqueness of positive radial solution follows from [12, Theorem 1.1 (i)]. Here we complement these results.

Corollary 1.4**.**

Let $B$ be a ball and $p,q>0$. Then,

1. (i)

Every pair of solutions $u,v\in C^{2}(B)\cap C(\overline{B})$ of (1.9) is radially symmetric and strictly decreasing; 2. (ii)

Under (1.10) with $pq\neq 1$, the problem (1.9) has a unique solution $u,v\in C^{2}(B)\cap C(\overline{B})$.

We stress that uniqueness for positive solutions in the superlinear case $pq>1$, under the extra condition $p,q\geq 1$, follows from [37, Theorem 1] combined with [12, Theorem 1.1 (i)]. The novelty here is the extension for $p,q>0$ in which one of them is strictly less than one, situation where differentiability or Lipschitz condition is no longer true.

The rest of the paper is organized as follows. In the preliminary Section 2 we introduce some notations and briefly recall the moving planes method. In Section 3 we develop the proofs of the main theorems. Section 4 is devoted to further discussions and applications. It includes results for equations involving Pucci’s extremal operators and a description for a continuum of solutions treated in [28], which comes from a class of problems with natural growth in the gradient.

2 Preliminaries and notations

Assume that $F_{i}:\overline{\Omega}\times{\mathbb{R}^{N}}\times\mathbb{S}^{N}(\mathbb{R})\rightarrow\mathbb{R}$ satisfies $(H_{1})$, for all $i\in\{1,\ldots,n\}$. In $(H_{1})$,

[TABLE]

are the Pucci’s extremal operators with ellipticity constants $0<\alpha_{i}\leq\beta_{i}$, $i\in\{1,\ldots,n\}$. See, for example, [7] for their properties. In particular, the condition $(H_{1})$ on the $X$ entry means that $F_{i}$ is a uniformly elliptic operator, uniformly continuous in $(p,X)$.

Throughout the text we denote

[TABLE]

where $\mu=\mu_{\sigma}$, with $\sigma=2\|w\|_{C^{1}(\overline{\Omega})}$, comes from hypothesis $(H_{2})$. Moreover, $\partial_{\nu}$ is the derivative in the direction of the interior unit normal vector. Along the text we will use the strong maximum principle and Hopf lemma for single equations involving Pucci’s operators in the form (2.1), which we write SMP and Hopf for short; for instance see [1].

Now we recall the definition of viscosity solution, understood in the $C$-viscosity sense; see also [7].

Definition 2.1**.**

We say that $\textbf{u}=(u_{1},\ldots,u_{n})\in(C(\Omega))^{n}$ is a viscosity subsolution $($supersolution$)$ of

$\mathcal{F}_{i}(x,\textbf{u},Du_{i},D^{2}u_{i})=0$*  in  $\Omega$,  $i=1,\ldots,n$,*

if, for each $i\in\{1,\cdots,n\}$, whenever $\phi\in C^{2}(B_{s}(x_{0}))$ is such that $u_{i}-\phi$ have a local maximum $($minimum$)$ at $x_{0}$, it follows $\mathcal{F}_{i}(x_{0},\textbf{u}(x_{0}),D\phi(x_{0}),D^{2}\phi(x_{0}))\geq 0\;\,(\mathcal{F}_{i}(x_{0},\textbf{u}(x_{0}),D\phi(x_{0}),D^{2}\phi(x_{0}))\leq 0)$.

Next we recollect the main ingredients and precise assumptions for the moving planes method in the $x_{1}$ axis direction. For the time being, consider $\Omega$ as a bounded $C^{2}$ domain. Assume that $\Omega$ is convex in the $x_{1}$ direction and symmetric with respect to the plane $x_{1}=0$, that is,

[TABLE]

Our tools are the parallel hyperplanes

[TABLE]

Let $R=\sup_{x\in\Omega}\,x_{1}$, where $x=(x_{1},\ldots,x_{N})$. For $\lambda<R$ we consider the right cap

[TABLE]

i.e. the part of the semiplane located on the right hand side of $T_{\lambda}$ which is in $\Omega$. Moreover, we denote by $A^{\lambda}$ the reflection of a set $A$ with respect to the plane $\{x_{1}=\lambda\}$.

We start decreasing $\lambda$ from $R$, by moving the plane $\{x_{1}=\lambda\}$ from right to left as far as $\{x_{1}=\lambda\}$ intersects $\overline{\Omega}$ with $\Sigma_{\lambda}^{\lambda}\subset\Omega$. From the usual index notation [20, 37], we denote

[TABLE]

The major challenge in the moving planes technique that goes back to Gidas, Ni and Nirenberg is to deal with the reflection of the boundary of $\Sigma_{\lambda}$, specifically the part of the boundary that is on $\partial\Omega$. Namely, $(\partial\Sigma_{\lambda}\cap\{x_{1}>\lambda\})^{\lambda}$ is contained within the domain $\Omega$ for $\lambda$ sufficiently close to $R$, due to the regularity of the domain.

As in [20], when we decrease the values of $\lambda$, it happens that $\Sigma_{\lambda}$ reaches a position in which at least one of the following situations occurs for the first time:

(I) $\Sigma_{\lambda}^{\lambda}$ becomes internally tangent to $\partial\Omega$ at some point which is not on $\{x_{1}=\lambda\}$;

(II) $\{x_{1}=\lambda\}$ reaches a position where it is orthogonal to $\partial\Omega$ at some point.

Such a value of $\lambda$ is denoted by $\Lambda_{1}$ (of course $\Lambda_{1}=0$ if $\Omega$ is a ball centered at [math]), that is,

[TABLE]

Note that $\Sigma_{\Lambda_{1}}^{\Lambda_{1}}\subset\Omega$, and the limiting position $\Lambda_{2}$ can be less than $\Lambda_{1}$. In general, $\Lambda_{2}\leq\Lambda_{1}$. Observe that, if $\Omega$ satisfies (2.2), then $R>0$, $\Lambda_{2}=0$; and if $\Lambda_{1}$ was positive, then it would happen at a point in which the plane $\{x_{1}=\Lambda_{1}\}$ is orthogonal to $\partial\Omega$.

The preceding difficulty never appears in the approach of Berestycki-Nirenberg for the moving planes method, since they do not need to take into account the behavior of the reflection of $\partial\Sigma_{\lambda}$, but only of $\Sigma_{\lambda}$ itself. This is possible through the maximum principle for small domains, which is applied in a neighborhood of $\partial\Omega$. Nevertheless, such an approach requires a Lipschitz condition on $f_{i}$ which we are not assuming.

In any case we define

[TABLE]

where $x^{\lambda}:=(2\lambda-x_{1},x^{\prime})$ is the reflection of the point $x$ with respect to the plane $\{x_{1}=\lambda\}$, for each $x=(x_{1},x^{\prime})\in\Omega\,$; here $x^{\prime}=(x_{2},\ldots,x_{N})\in\mathbb{R}^{N-1}$.

Regarding the $x_{1}$ direction, as in [11] we denote $\bar{p}=(-p_{1},p_{2},\ldots,p_{N})$, and $\bar{X}$ as the matrix with entries $\epsilon_{\iota j}X_{\iota j}$, where $\epsilon_{11}=1$, $\epsilon_{\iota j}=1$ if $\iota,j\geq 2$, and $\epsilon_{1j}=\epsilon_{j1}=-1$ if $j\neq 1$, for any $p=(p_{1},\ldots,p_{N})\in{\mathbb{R}^{N}}$ and for any symmetric matrix $X=(X_{\iota j})_{\iota,j}\in\mathbb{S}^{N}(\mathbb{R})$. Observe that $X$ and $\bar{X}$ have the same eigenvalues. We then assume as in [2], instead of $(H_{0})$,

• $(\widetilde{H}_{0})$

$F_{i}\,(y_{1},x^{\prime},\bar{p},\bar{X})+f_{i}\,(y_{1},x^{\prime},\textbf{u},\bar{p})\geq F_{i}\,(x,p,X)+f_{i}\,(x,\textbf{u},p)$, for all $p,X$, $\textbf{u}=(u_{1},\ldots,u_{n})$, and for $x=(x_{1},x^{\prime})\in\Omega$ such that $y_{1}<x_{1}$ with $y_{1}+x_{1}>0$.

Notice that $(\widetilde{H}_{0})$ itself comprises monotonicity in the $x$ entry with respect to the $x_{1}$ direction; so we do not need to assume a version of $(H_{3})$ in the $x_{1}$ direction.

Notice that $x_{1}>x_{1}^{\lambda}>-x_{1}$ for all $x\in\Sigma_{\lambda}$ when $\lambda>\Lambda_{2}$. Thus, it follows from $(\widetilde{H}_{0})$ that, for any solution $(u_{1},\ldots,u_{n})$ of (4.16), $u_{i}^{\lambda}$ satisfies the following inequality

[TABLE]

Further, as pointed out in [20], we need the following hypothesis on $f_{i}(\cdot,0,\ldots,0)$ in the nonradial case.

• $(H_{7})$

For each $i\in\{1,\ldots,n\}$, on $\partial\Omega\cap\{x_{1}>0\}$ we have either $f_{i}\,(\cdot,0,\ldots,0)\geq 0$ or $f_{i}\,(\cdot,0,\ldots,0)<0$.

Note that such condition is trivially satisfied in the radial scenario since $f(x,0,\ldots,0)$ is constant on $\partial\Omega$. Moreover, set ($\widetilde{H}$) as

[TABLE]

To finish the section, we observe that assumption $(H_{5})$ on $f_{i}$ is equivalent to $c_{ij}\geq 0$ for all $i\neq j$, in which the function $c_{ij}$ is defined as

[TABLE]

where $\textbf{w}^{j}=\textbf{w}^{j}(\textbf{u},h)=(w_{1},\ldots,w_{n})$ with $w_{j}=u_{j}+h$, $w_{l}=u_{l}$ if $l\neq j$ and $\textbf{u}=(u_{1},\ldots,u_{n})$. We also denote the standard orthonormal basis in ${\mathbb{R}^{N}}$ as $\{e_{l}\}_{1\leq l\leq N}$.

Remark 2.2**.**

Notice that, for $\lambda>\Lambda_{2}$, the regularity of the domain implies the existence of some points in the reflection of the boundary portion $\partial\Sigma_{\lambda}\cap\partial\Omega$ which lie in $\Omega$. So, at such points we have $U_{i}^{\lambda}>0$ for all $i\in\{1,\ldots,n\}$. In particular, we can never have $U_{i}^{\lambda}\equiv 0$ in $\Sigma_{\lambda}$ for some $i$ in this case. If, moreover, $\lambda>\Lambda_{1}$, then $(\partial\Sigma_{\lambda}\cap\{x_{1}>\lambda\})^{\lambda}\subset\Omega$.

3 Proofs of the main theorems

We start with a symmetry result that comprises a class of domains for which symmetry can be obtained in one direction without additional hypotheses on the operators $F_{i}$, for example if $\Omega$ is an ellipse.

Proposition 3.1**.**

Let $\Omega$ be a $C^{2}$ domain satisfying (2.2), where situation $\mathrm{(II)}$ in the preceding section does not occur for any $\lambda>0$. Assume ($\widetilde{H}$), for all $1\leq i\leq n$. Let $(u_{1},\ldots,u_{n})$ be a solution of (1.4), where $u_{i}$’s are $C^{2}(\overline{\Omega})$ functions. Then $u_{i}$ is symmetric with respect to the plane $x_{1}=0$, and $\partial_{x_{1}}u<0$ for any $x_{1}>0$, for all $1\leq i\leq n$.

Notice that a ball is the simplest example of a domain satisfying the hypotheses of Proposition 3.1. Furthermore, observe that this proposition and $(H_{0})$ produce the radial symmetry and monotonicity of Theorem 1.1. Indeed, concerning an arbitrary direction $w$, we reduce it to the case of the $x_{1}$ direction via rotation $x\mapsto xQ$, where $Q$ is a $N\times N$ orthogonal matrix such that $wQ=e_{1}$. The symmetry of $u_{i}$ in the direction $w$ follows from the symmetry of $W_{i}(x):=u_{i}(xQ)$ with respect to the plane $x_{1}=0$.

If, moreover, the operators $F_{i}$, $1\leq i\leq n$, are continuously differentiable in the entry $X\in\mathbb{S}^{N}(\mathbb{R})$, the following more general symmetry result in one direction can be accomplished.

Theorem 3.2**.**

Let $\Omega$ be a bounded $C^{2}$ domain satisfying (2.2). Assume that $F_{i}\,(x,p,X)$ is continuously differentiable in $X$, and that $f_{i}$ and $F_{i}$ verify ($\widetilde{H}$), for all $1\leq i\leq n$. Let $(u_{1},\ldots,u_{n})$ be a solution of (1.4), where $u_{i}$’s are $C^{2}(\overline{\Omega})$ functions. Then, for each $1\leq i\leq n$, $u_{i}$ is symmetric in the $x_{1}$ direction and $\partial_{x_{1}}u_{i}<0$, for all $x\in\Omega$ with $x_{1}>0$.

For ease of notation, we provide a proof in the case $n=2$, with $u_{1},u_{2}$, $f_{1},f_{2}$, and $F_{1},F_{2}$ are replaced by $u,v$, $f,g$, and $F,G$, respectively. We indicate the respective changes that appear from the incorporation of other components when necessary. The system is rewritten as

[TABLE]

For the decomposition in $(H_{4})$ we just rewrite $f=f_{1}+f_{2}$ and $g=g_{1}+g_{2}$.

Then we consider $U^{\lambda}:=u^{\lambda}-u,\;V^{\lambda}:=v^{\lambda}-v,$ for $u^{\lambda}(x)=u(x^{\lambda})$, $v^{\lambda}(x)=v(x^{\lambda})$.

We will need the following lemma, which is a consequence of SMP and Hopf. It concerns the domain $\Sigma_{\lambda}$ when $\lambda>\Lambda_{2}$, and solutions of (3.5).

Lemma 3.3**.**

$($SMP and Hopf in $\Sigma_{\lambda})$ Let $\lambda>\Lambda_{2}$ and let $u,v$ be a pair of solutions of (3.5). Assume $U^{\lambda}$ and $V^{\lambda}$ nonnegative in $\Sigma_{\lambda}$ and ($\widetilde{H}$). Then $U^{\lambda},V^{\lambda}>0$ in $\Sigma_{\lambda}$ and $u_{x_{1}},\,v_{x_{1}}<0$ on $T_{\lambda}$.

Proof.

Using $(H_{1})$, (2.1), (2.3), and $(H_{2})$ we obtain

[TABLE]

since $f_{2}$ is nondecreasing in $u$. Here $d_{f}$ is the uniform Lipschitz constant of $t\mapsto f_{1}(x,t,v,p)$ for $t\in[0,\sup_{\Omega}u]$, $p\in\overline{B}_{\sigma}(0)$ with $\sigma=2\|u\|_{C^{1}(\overline{\Omega})}$ and $c_{12}$ comes from (2.4). Using further $c_{12}(x,V^{\lambda},Du^{\lambda})V^{\lambda}\geq 0$, we derive $\mathcal{\widetilde{L}}^{-}[\,U^{\lambda}\,]\leq d_{f}\,U^{\lambda}$, see (2.1). Then, SMP entails either $U^{\lambda}>0$ in $\Sigma_{\Lambda}$ or $U^{\lambda}\equiv 0$ in $\Sigma_{\lambda}$. Thus, by Remark 2.2, $U^{\lambda}>0$ in $\Sigma_{\lambda}$. Since the interior unit normal vector on $\partial\Sigma_{\lambda}\cap T_{\lambda}$ is $\nu=e_{1}$, Hopf yields $0<\partial_{\nu}\,U^{\lambda}=-2\,u_{x_{1}}$ on $T_{\lambda}$.

The proof of $V^{\lambda}>0$ in $\Sigma_{\lambda}$ is analogous, by considering ($\widetilde{H}$) for $g$, and $d_{g}$ as the uniform Lipschitz constant of $s\mapsto g_{1}(x,u,s,p)$ for $s\in[0,\sup_{\Omega}v]$, $p\in\overline{B}_{\sigma}(0)$. ∎

Proof of Proposition 3.1..

We split the proof in two steps.

Step 1: Start moving the planes.

Notice that the first component of the interior unit normal vector at a boundary point $x$, denoted by $\nu_{1}(x)$, is negative for any $x\in\partial\Omega$ which is close to points of the form $(R,x^{\prime})$, where $R=\sup_{x\in\Omega}x_{1}$ from Section 2. Fix $x_{0}=(R,x_{0}^{\prime})$ and $\varepsilon>0$ such that this property remains true for all points on $\partial\Omega\cap B_{\varepsilon}(x_{0})$.

By taking a smaller $\varepsilon$ if necessary (independent of $u$), we are going to show that

[TABLE]

Repeating the same for $v$, it will imply $U^{\mu},V^{\mu}>0$ in $\Sigma_{\mu}$, $\mu\in(\lambda,R)$, for values of $\lambda$ close to $R$.

Case 1. Assume first $f(x,0,0,0)\geq 0$, for all $x\in\Omega$. In this case, since $F(x,0,0)\equiv 0$,

[TABLE]

Then $\mathcal{\widetilde{L}}^{-}[u]\leq d_{f}u$ and Hopf give rise to $\partial_{\nu}u>0$ on $\partial\Omega$; by a covering argument we obtain (3.7).

*Case 2. Suppose that there exists some $\tilde{x}\in\Omega$ such that $f(\tilde{x},0,0,0)<0$. *

In this case, by assuming that (3.7) is not true, there exists a sequence of points $z_{k}$ in $\Omega$ converging to $x_{0}$ such that $u_{x_{1}}(z_{k})\geq 0$. Observe that $u>0$ in $\Omega$ and $u=0$ on $\partial\Omega$ imply $u_{x_{1}}(x_{0})\leq 0$. Hence, $u_{x_{1}}(x_{0})=0$. This and the fact that the gradient of $u$ is parallel to the unit normal vector at $x_{0}$ yield $Du(x_{0})=0$.

For each $k$, the segment in the positive $x_{1}$ direction from $z_{k}$ intersects $\partial\Omega\cap B_{\varepsilon}(x_{0})$ at a point $y_{k}$ where $u_{x_{1}}(y_{k})\leq 0$. The mean value theorem gives $\xi_{k}$ such that $0\leq u_{x_{1}}(z_{k})-u_{x_{1}}(y_{k})=u_{x_{1}x_{1}}(\xi_{k})\,(z_{k,1}-y_{k,1}),$ from where $u_{x_{1}x_{1}}(\xi_{k})\leq 0$, and so $u_{x_{1}x_{1}}(x_{0})\leq 0$ in the limit.

Fix an arbitrary $t_{0}\in T_{x_{0}}$. As in [37], let $T_{x}$ be the tangent space to $\partial\Omega$ at $x$, and let $x(s)$ be a fixed path on $\partial\Omega$ such that $x(0)=x_{0}$, $\dot{x}(0)=t_{0}$. From $u(x(s))=0$ for all $s$, it follows that $Du(x(s))\cdot\dot{x}(s)=0$ for all $s$ as well. Furthermore,

[TABLE]

Take some $t_{1}\in T_{x_{0}}$ and let $t(s)$ be a path with $t(s)\in T_{x(s)}$ and $t(0)=t_{1}$. From $Du(x(s))\cdot t(s)=0$ for all $s$, by differentiating it and using $Du(x_{0})=0$, yields

[TABLE]

Next, since the function $\varphi(s):=Du(x(s))\cdot\nu(x(s))=\partial_{\nu}u(x(s))$ has a minimum equal to zero at [math], then $\varphi^{\prime}(0)=0$, from where $\nu(x_{0})\cdot(D^{2}u(x_{0})\cdot t_{0})=0$. This and (3.8) imply that $D^{2}u(x_{0})\cdot t_{0}=0$, where $t_{0}\in T_{x_{0}}$ is arbitrary. Consequently, [math] is an eigenvalue of order $N-1$ for $D^{2}u(x_{0})$. Namely, let $\{e,0,\ldots,0\}$ be the spectrum of $D^{2}u(x_{0})$.

We claim that $e>0$. Indeed, if $e\leq 0$, then $\mathcal{M}^{+}_{1}(D^{2}u(x_{0}))=e\alpha_{1}\leq 0$. Hence

[TABLE]

Consider the point $\tilde{z}$ in which the line segment on the $x_{1}$ direction from $\tilde{x}$ hits $\partial\Omega$. By monotonicity,

[TABLE]

Clearly, (3.9) and (3.10) contradict the hypothesis $(H_{5})$. Thus, the claim $e>0$ is proved.

Now, a basis of $\mathbb{R}^{N}$ composed of $\nu(x_{0})$ and $\{a_{l}\}_{1\leq l\leq N-1}$ (orthonormal basis of $T_{x_{0}}$) applied to the above calculations ensures that $D^{2}u(x_{0})=(e\,\nu_{i}\nu_{j})_{ij}$. In particular, $u_{x_{1}x_{1}}(x_{0})=e\,\nu_{1}(x_{0})\nu_{1}(x_{0})$ is positive, which contradicts $u_{x_{1}x_{1}}(x_{0})\leq 0$. We so conclude (3.7) and Step 1.

Actually, Step 1 is completely independent and it only uses the regularity of the solutions up to the boundary, along with the regularity of the boundary itself. That is why we kept it in terms of $\Lambda_{2}$. Recall $\Lambda_{2}=\Lambda_{1}=0$ under hypotheses of Proposition 3.1.

Therefore, by Step 1, it is well defined the following quantity

[TABLE]

By continuity, $U^{\Lambda}\geq 0$ and $V^{\Lambda}\geq 0$ in $\Sigma_{\Lambda}$.

Step 2: Stop moving the planes at zero.

The goal is to show that $\Lambda=0$. Suppose $\Lambda>0$ in order to obtain a contradiction. Thus, by Lemma 3.3, we have that $U^{\Lambda},V^{\Lambda}>0$ in $\Sigma_{\Lambda}$ and $u_{x_{1}},v_{x_{1}}<0$ on $T_{\Lambda}$. Furthermore, there exists some $\varepsilon>0$ such that

[TABLE]

Indeed, $u_{x_{1}},v_{x_{1}}<0$ on $T_{\Lambda-\varepsilon}\cap K$, for any compact $K\subset\Omega$, with $\varepsilon=\varepsilon(K)$. On a neighborhood of the boundary points $x\in\partial\Omega\cap\{x_{1}=\Lambda-\varepsilon\}$, we can apply exactly the same argument used to derive (3.7). This ensures (3.11).

Consider, then, a sequence of points $x_{k}\in\Sigma_{\lambda_{k}}$, with $\lambda_{k}\in(\Lambda-\varepsilon,\Lambda)$, for some $\varepsilon\in(0,\Lambda)$, $\lambda_{k}\rightarrow\Lambda$ as $k\rightarrow\infty$, such that, for each $k\in\mathbb{N}$, either $U^{\lambda_{k}}(x_{k})\leq 0$ or $V^{\lambda_{k}}(x_{k})\leq 0$.

By passing to a subsequence, say $x_{k}\rightarrow z\in\overline{\Sigma}_{\Lambda}$ and $U^{\lambda_{k}}(x_{k})\leq 0$ (at least one of them verifies this for infinite $k$’s). Thus, $U^{\Lambda}(z)\leq 0$. Now, since $U^{\Lambda}>0$ in $\Sigma_{\Lambda}\cup(\partial\Sigma_{\Lambda}\cap\{x_{1}>\Lambda\})$, we must have $z\in\partial\Sigma_{\Lambda}\cap\{x_{1}=\Lambda\}$ and, from (3.11), $u_{x_{1}}(z)\leq 0$. Further, since the line segment between $x_{k}$ and $x^{\lambda_{k}}_{k}$ is contained in $\Omega$, the mean value theorem yields $y_{k}\in\Omega$ such that

[TABLE]

Hence $u_{x_{1}}(y_{k})\geq 0$. Here $y_{k}\rightarrow z$, since both $x_{k}$ and $x^{\lambda_{k}}_{k}$ converge to $z=z^{\Lambda}$, and this contradicts (3.11). ∎

For the proof of Theorem 3.2 we recall the following lemma which is needed in the usual treatment of Serrin’s corner lemma for nonlinear operators, as in [20, 33, 35].

Lemma 3.4**.**

Assume that $F_{i}(x,p,X)$ is $C^{1}$ in $X$ and satisfies $(\widetilde{H}_{0})$, $(H_{1})$, for some $i$. Let $\lambda$ be such that $U_{i}^{\lambda}>0$ in $\Sigma_{\lambda}$. Then $\partial^{2}_{s}\,U_{i}^{\lambda}(z)>0$ for any direction $s$ that enters in $\Sigma_{\lambda}$ at $z$ nontangentially, at any point $z\in\partial\Omega\cap\{x_{1}=\lambda\}$ in which $T_{\lambda}$ is orthogonal to $\partial\Omega$.

Proof.

Set $p=p(x)=Du_{i}(x)$. Then by splitting

[TABLE]

and using calculations from equation (3), $U_{i}^{\lambda}$ becomes a solution of a linear equation in the form

[TABLE]

where $b$ is bounded, and $A=(a_{jk})_{jk}$ has continuous entries in $\overline{\Sigma}_{\lambda}$. Precisely, $b(x)=DU_{i}^{\lambda}(x)$ $(\mu+{\gamma}/|DU_{i}^{\lambda}(x)|)$ if $|DU_{i}^{\lambda}(x)|\neq 0$, and $b(x)=0$ otherwise;

[TABLE]

Since $F_{i}(x,p,X)=F_{i}(x,p,\bar{X})$, it follows that $\partial_{X_{1k}}F_{i}(x,p,X)+\partial_{X_{1k}}F_{i}(x,p,\bar{X})=0$ for $k>1$. Hence, $a_{1k}=0$ on $T_{\lambda}\cap\partial\Sigma_{\lambda}$ for all $k>1$. An application of Serrin’s lemma [33, Lemma 2], at any point $z\in\partial\Omega\cap\{x_{1}=\lambda\}$ in which $T_{\lambda}$ is orthogonal to $\partial\Omega$, yields

[TABLE]

for any direction $s$ that enters in $\Sigma_{\lambda}$ at $z$ nontangentially. At such $z$, however, the functions $u_{i}^{\lambda}$ and $u_{i}$ have the same normal derivative to $\partial\Omega$, and zero tangential derivatives, thus $\partial_{s}U_{i}^{\lambda}(z)=0$. Therefore, $\partial^{2}_{s}\,U_{i}^{\lambda}(z)>0$. ∎

Proof of Theorem 3.2..

The start of moving planes is identical to Step 1 in the previous proof, since for $\lambda$ sufficiently close to $R$ the domain does not reach position $\Lambda_{1}$. For Step 2, we need to take situation (II) into account. Assume $\Lambda>0$ in manner to achieve a contradiction. By Lemma 3.3 we have $U^{\Lambda}>0$ in $\Sigma_{\Lambda}$ and $u_{x_{1}}<0$ on $T_{\Lambda}$. Analogously to the previous Step 2, since $\Lambda_{2}=0$ is not the infimum $\Lambda$, there are sequences $\lambda_{k}\in(0,\Lambda)$, $\lambda_{k}\rightarrow\Lambda$, and $x_{k}\in\Sigma_{\lambda_{k}}$ such that $u(x_{k})\geq u(x_{k}^{\lambda_{k}})$. We may suppose $x_{k}\rightarrow z$ in $\overline{\Sigma}_{\Lambda}$. Then, $U^{\Lambda}(z)\leq 0$, and so $z\in\partial\Sigma_{\Lambda}$.

Case 1: $z\in\{x_{1}=\Lambda\}$.

Notice that the line segment between $x_{k}$ and $x_{k}^{\lambda_{k}}$ lies in $\Omega$. Then, by the mean value theorem, there exists $y_{k}$ on it such that $0\leq u(x_{k})-u(x_{k}^{\lambda_{k}})=2u_{x_{1}}(y_{k})(x_{k,1}-\lambda_{k})$, from where $u_{x_{1}}(y_{k})\geq 0$. In the limit, $u_{x_{1}}(z)\geq 0$, and thus $z\in\partial\Omega$. Using again that $\partial_{\nu}u\geq 0$ on $\partial\Omega$, we have $u_{x_{1}}(z)\leq 0$, thus $u_{x_{1}}(z)=0$.

We claim that $T_{\Lambda}$ is orthogonal to $\partial\Omega$ at $z$. In fact, otherwise we would necessarily have $\nu_{1}(z)<0$, which in particular is enough to apply Hopf’s lemma to $u$ (at $z\in\partial\Omega$) in order to conclude $u_{x_{1}}(z)<0$; but this contradicts $u_{x_{1}}(z)=0$.

Now we are in position of applying Serrin’s lemma. By Lemma 3.4, $\partial^{2}_{s}\,U^{\Lambda}(z)>0$, for any direction $s$ that enters in $\Sigma_{\Lambda}$ at $z$ nontangentially. Next we compute it for the direction $s=(1/\sqrt{2},0,\ldots,0,-1/\sqrt{2})$ properly chosen. Without loss of generality, we consider the interior normal as $\nu(z)=-e_{N}$. Then, at $z$,

[TABLE]

Using that $u^{\Lambda}_{x_{1}x_{1}}=u_{x_{1}x_{1}}$, $u^{\Lambda}_{x_{N}x_{N}}=u_{x_{N}x_{N}}$, and $u^{\Lambda}_{x_{1}x_{N}}=u_{x_{1}x_{N}}$ at $z$, we obtain from (3.12) that $u_{x_{1}x_{N}}>0$ in a neighborhood of $z$. We consider the segment $I_{k}$ from $x_{k}$ to the point $z_{k}$ where it hits $\partial\Omega$ in the $e_{N}$ direction. Integration on $I_{k}$ for large $k$ gives $u_{x_{1}}(z_{k})>0$. However, from $\partial_{\nu}u\geq 0$ on $\partial\Omega$ we have $u_{x_{1}}(z_{k})\leq 0$ for large $k$, which yields a contradiction.

Case 2: $z\in\{x_{1}>\Lambda\}$.

The first thing to note is that $z^{\Lambda}\in\partial\Omega$, since $U^{\Lambda}(z)$ would be positive otherwise. Hence, since $\Lambda>0$, the interior normals $\nu(z)$ and $\nu(z^{\Lambda})$ coincide, and these are orthogonal to $e_{1}$. As in Case 1, w.l.g. say $\nu(z)=\nu(z^{\Lambda})=-e_{N}$. Moreover, since $U^{\Lambda}(z)=0$, we can apply Hopf to the function $U^{\Lambda}$ in the domain $\Sigma_{\Lambda}$, from where $U^{\Lambda}_{x_{N}}(z)<0$.

Observe that, under our contradiction assumption $\Lambda>0$ on the symmetric convex domain $\Omega$, we have $\Sigma^{\Lambda}_{\Lambda}\subset\Omega$, and so the segment $I_{k}$ in the $e_{N}$ direction from $x_{k}$ to $\partial\Omega$ is not longer than $I_{k}^{\lambda_{k}}$, i.e. $|I_{k}|\leq|I^{\lambda_{k}}_{k}|$. By the mean value theorem applied to $u$, there exists $\xi_{k}\in I_{k}$, $\tilde{\xi}_{k}\in I_{k}^{\lambda_{k}}$ such that $u_{x_{N}}(\xi_{k})|I_{k}|=-u(x_{k})\leq-u(x_{k}^{\lambda_{k}})=u_{x_{N}}(\tilde{\xi}_{k})|I_{k}^{\lambda_{k}}|$, from where $u_{x_{N}}(x_{k})\leq u_{x_{N}}(x_{k}^{\lambda_{k}})$. Passing to limits we obtain $u_{x_{N}}(z)\leq u_{x_{N}}(z^{\Lambda})$, which contradicts $U^{\Lambda}_{x_{N}}(z)<0$. ∎

Next, Theorem 1.2 is a consequence of the proof of Theorem 1.1 and [37, Theorem 2]. The proof follows the main ideas there, which we include for the sake of completeness.

Proof of Theorem 1.2..

Our aim is to prove that $\Omega$ is symmetric with respect to the plane $x_{1}=\lambda_{1}$. Then the same argument applied to any other direction will imply that $\Omega$ must be a ball, and so the symmetry is obtained via Theorem 1.1. Observe that exactly the same proof of Proposition 3.1, by replacing [math] by $\Lambda_{1}$ without the symmetry assumption (2.2) on $\Omega$, gives $\Lambda=\Lambda_{1}$, where

[TABLE]

and for $\Lambda_{1}$ as defined in Section 2. This is due to the lack of the situation (II) for all $\lambda\in(\Lambda_{1},R)$.

Next, by continuity, we have $U_{i}^{\Lambda_{1}}\geq 0$ in $\Sigma_{\Lambda_{1}}$ for all $i\in\{1,\ldots,n\}$. By SMP, either $U_{i}^{\Lambda_{1}}>0$ or $U_{i}^{\Lambda_{1}}\equiv 0$ in $\Sigma_{\Lambda_{1}}$, for each $i$. If the latter occurs for all $i\in\{1,\ldots,n\}$, then $\Omega$ is symmetric with respect to $x_{1}=\Lambda_{1}$, and the proof is finished.

In order to obtain a contradiction, assume that $U_{i}^{\Lambda_{1}}>0$ in $\Sigma_{\Lambda_{1}}$ for some $i\in\{1,\ldots,n\}$. We claim that, in this case, $\Lambda_{1}$ is accomplished through situation (II), that is, at a point in which the plane $\{x_{1}=\Lambda_{1}\}$ reaches an orthogonal position to $\partial\Omega$. Indeed, if situation (I) happened, then we would have $x_{0}^{\Lambda_{1}}\in\partial\Omega$, for some $x_{0}\in\partial\Sigma_{\Lambda_{1}}\cap\{x_{1}>\Lambda_{1}\}$. So, $U_{i}^{\Lambda_{1}}(x_{0})=0$ and, by Hopf applied to $U_{i}^{\Lambda_{1}}$, would yield $\partial_{\nu}U_{i}^{\Lambda_{1}}(x_{0})>0$. However, this contradicts the assumption on the overdetermined problem, namely $\partial_{\nu}U_{i}^{\Lambda_{1}}(x_{0})=\partial_{\nu}u_{i}(x_{0})-\partial_{\nu}u_{i}^{\Lambda_{1}}(x_{0})=0$, and the claim is proved. Now Serrin’s argument [33, (a) in p. 307] gives us that $U^{\Lambda_{1}}_{i}$ has a zero of second order at $x_{0}$. Finally, a contradiction is established with Serrin’s lemma as in Lemma 3.4 where $\lambda=\Lambda_{1}$. ∎

Notice that if the domain satisfies (2.2) and it does not have a point for which case $\mathrm{(II)}$ from Section 2 occurs, then the proofs of Theorems 3.2 and 1.2 carry out without using Serrin’s lemma. In particular, they hold true without any differentiability hypothesis on $F_{i}$. In other words, this comprises a reasonable class of domains in which differentiability can be dropped. For such domains it is also possible to obtain a restatement of Theorem 1.3 for viscosity solutions which are continuously differentiable up to the boundary. This is the content of the next proof.

Proof of Theorem 1.3.

As in Theorem 1.1, we only need to prove a version of Proposition 3.1 with respect to viscosity $C^{1}(\overline{\Omega})$ solutions. We first claim that $u_{i}$ is a viscosity solution of $F_{i}(x,Du_{i},D^{2}u_{i})+\bar{f}_{i}(x)=0$ in $\Omega$, where $\bar{f}_{i}(x):=f_{i}(x,u_{1}(x),\ldots,u_{n}(x),Du_{i}(x))\in C(\overline{\Omega})$.

Let us prove the subsolution case; for the supersolution it is analogous. Assuming the contrary, there exist some $x_{0}\in\Omega$ and $\phi\in C^{2}(B_{s}(x_{0}))$ such that $u_{i}-\phi$ has a local maximum at $x_{0}$ but $F_{i}(x_{0},D\phi(x_{0}),D^{2}\phi(x_{0}))-\bar{f}_{i}(x_{0})<0$. By the definition of $u_{i}$ being a viscosity subsolution of the equation in (1.4), we have that $F_{i}(x_{0},D\phi(x_{0}),D^{2}\phi(x_{0}))+f_{i}(x_{0},u_{1}(x_{0}),\ldots,u_{n}(x_{0}),D\phi(x_{0}))\geq 0$. Since $u_{i}-\phi\in C^{1}(B_{s}(x_{0}))$ has a local maximum at $x_{0}$, we have $D(u_{i}-\phi)(x_{0})=0$. Thus the last two inequalities produce a contradiction.

Similarly we prove that $w_{i}=u_{i}^{\lambda}$ is viscosity supersolution of $F_{i}(x,w_{1},\ldots,w_{n},Dw_{i},D^{2}w_{i})+\bar{f}_{i}^{\lambda}(x)=0$ in $\Omega$ with $\bar{f}^{\lambda}_{i}(x):=f_{i}(x,u_{1}^{\lambda}(x),\ldots,u_{n}^{\lambda}(x),Du_{i}^{\lambda}(x))$, for $i=1,\ldots,n$.

Consider for simplicity $n=2$, $u_{1}=u$ and $u_{2}=v$. For the r.h.s. $\bar{f}_{i}^{\lambda}-\bar{f}_{i}$ we proceed as in (3), which produces an extra term of first order $\mu|DU^{\lambda}|$. Meanwhile, the l.h.s. is treated in the viscosity sense by using Proposition 2.1 in [11]. In other words, $w=U^{\lambda}$ is a viscosity solution of $-\mathcal{L}^{-}[w]\geq-d_{f}U_{\lambda}-\mu|DU^{\lambda}|$ in $\Sigma_{\lambda}$.

Next we infer that such term in the r.h.s. can be retreated as being part of the Pucci operator in the l.h.s., that is, $U^{\lambda}$ is a viscosity solution of $\widetilde{\mathcal{{L}}}^{-}[\,U^{\lambda}\,]\leq d_{f}U^{\lambda}$. Indeed, this is again a consequence of the differentiability of $u$ and the definition of viscosity solution.

Hence we carry on the rest of the proof of Proposition 3.1 in the same way by using SMP and Hopf for viscosity solutions in order to establish the desired symmetry. ∎

4 Discussion and further applications

We stress that both works [37] and [34] use the differentiability of $f_{i}$ in order to obtain radial symmetry for systems. In fact, a typical example already discussed in Section 1 is the Lane-Emden system (1.9), in which it is clearly important to contemplate cases where exponents can be less than one.

Alternatively, there are other relevant applications to systems with nondifferentiable terms. For instance, in [8, 9] the authors developed an analysis about the behavior between different species $u$ and $v$ that cohabit in $B$, in particular from the following systems

[TABLE]

where $N=2,3$ and $\beta\in\mathbb{R}$. The system (4.4) can be treated variationally as long as the right hand sides are written as $F_{u}(u,v)$ and $F_{v}(u,v)$ respectively, where $4F(u,v)={u^{4}}+{v^{4}}-2{\beta}u^{2}v^{2}.$ From their study it can be derived the more general problem

[TABLE]

where $r+s=p$, for $2<p<2^{*}$, $r,s>1$, $N\geq 1$. Since the involved functions are not necessarily differentiable, our results provide (new) radial symmetry for positive solutions of (4.8) in the harmonious case, that is, with $\beta<0$. On the other side, in a competitive scenario, with $\beta>0$, cooperativity is lost and our results do not apply. More than that, symmetry breaking occurs, see [36, Remark 5.4], due to the particular segregation phenomenon in the limit $\beta\rightarrow+\infty$ described in [8, 9].

Now we consider the following Lane-Emden type system involving Pucci’s operators

[TABLE]

where $p,q>0$, $pq\neq 1$, $\mathcal{M}_{i}$ can be either $\mathcal{M}^{+}_{\alpha_{i},\beta_{i}}$ or $\mathcal{M}^{-}_{\alpha_{i},\beta_{i}}$ in a ball.

In a lot of cases, a Pucci’s extremal operator can feature the essence of the Laplacian. For example, for a single equation (that is, (4.13) with $u=v$, $\mathcal{M}_{1}=\mathcal{M}_{2}$, $p=q$), existence of a unique classical solution was extended to Pucci’s operator in [29] with locally Lipschitz nonlinearities, while existence in the sublinear case was first established in [30].

However, criticality relations are now given in terms of the ellipticity coefficients. Those are related to critical exponents in Liouville type results, but not completely understood even in the scalar case; see [10, 18, 30]. Moreover, they can be much more complicated in the case of a system. As far as existence is concerned, define $\rho_{i}=(\alpha_{i}/\beta_{i})\pm 1$ and $N_{i}=\rho_{i}(N-1)+1$, $i=1,2$. For instance, in [31] it was proved that there exists a positive classical solution of (4.13) if $p,q\geq 1$, $pq>1$, and $2(p+1)/(pq-1)\geq N_{1}-2$ or $2(q+1)/(pq-1)\geq N_{2}-2$, in a smooth bounded domain.

On the other hand, it is known that, in many cases, a radialization of the problem can greatly simplify the operators, specially if we are dealing with Pucci’s operators. When radial assumptions on the domain and on the solutions are imposed, sometimes it is possible to go much further; see for instance [18, 19] and references therein.

We stress that, regardless whether or not solutions exist, in this work we are concerned just with their radial symmetry – in the sense that, if a solution exists then it is radial. In this direction, our Theorem 1.1 provides radial symmetry to solutions of (4.13). In addition, it says that we can focus on establishing the unknown properties for positive solutions that are radial in nature.

Finally, we discuss some properties of solutions of the following family of problems

[TABLE]

where $r=|x|$, $h\in L^{\infty}(\Omega)$, $F$ satisfies $(H_{0})$, $(H_{1})$, with $c\gneqq 0$, $\mu(r)\in[\mu_{1},\mu_{2}]$, for some $\mu_{1},\mu_{2}>0$.

This class of problems is important in applications, more recently in control theory and mean field games. Moreover, it is of theoretical independent interest since this type of gradient dependence is invariant under diffeomorphic changes of variable and function, namely $x$ and $u$. In particular, symmetry properties might play an important role in the qualitative analysis of the set of solutions.

Now, assume that $c(r),\mu(r),h(r)$ are continuous functions in $r$ in order to treat the problem in the $C$-viscosity sense from Definition 2.1. Observe that viscosity solutions of (4.16) belong to $C^{1,\alpha}(\overline{\Omega})$ due to [27, Theorem 1.1]. Further, we have from [28] the following multiplicity result. Recall that strong solutions belong to some Sobolev space $W^{2,p}$ and satisfy the equation a.e. Here $p>N$ and so $W^{2,p}(\Omega)\subset C^{1}(\overline{\Omega})$.

Proposition 4.1**.**

Under the preceding hypotheses, assume that the problem $(P_{0})$ has a strong solution $u_{0}\geq 0$, with $cu_{0}\gneqq 0$. Then every nonnegative viscosity solution of (4.16) with $\lambda>0$ satisfies $u>u_{0}$. Moreover, there exists $\bar{\lambda}\in(0,+\infty)$ such that for every $\lambda\in(0,\bar{\lambda})$, the problem (4.16) has at least two nontrivial strong solutions with $u_{\lambda,1}\leq u_{\lambda,2}\,$, where $u_{0}<u_{\lambda_{1},1}<u_{\lambda_{2},1}$ if $\,0<\lambda_{1}<\lambda_{2}\,$, $u_{\lambda,1}\rightarrow u_{0}$ in $C^{1}(\overline{\Omega})$, and $\max_{\overline{\Omega}}u_{\lambda,2}\rightarrow+\infty$ as ${\lambda\rightarrow 0^{+}}$. The problem $(P_{\bar{\lambda}})$ has a unique strong solution; and for $\lambda>\bar{\lambda}$, the problem (4.16) has no nonnegative solution.

Thus, the continuum of solutions illustrated in Figure 1 consists of radial and strictly decreasing functions by Theorem 1.3, provided $c,\mu,h$ are nonincreasing with the radius.

Acknowledgment. Ederson Moreira dos Santos was partially supported by CNPq grant 307358/2015-1, and Gabrielle Nornberg was supported by FAPESP grant 2018/04000-9.