On qualitative properties of solutions for elliptic problems with the $p$-Laplacian through domain perturbations

TL;DR

This paper investigates how the critical energy levels of solutions to a p-Laplacian elliptic problem depend on domain shape perturbations, establishing formulas and geometric properties that influence solution symmetry and energy levels.

Contribution

It introduces Hadamard-type formulas for critical levels and characterizes domain shapes that maximize these levels, revealing nonradial solutions in symmetric domains.

Findings

Critical levels attain maximum in concentric spherical annuli.

Least energy nodal solutions are nonradial in balls and concentric annuli.

Domain shape significantly influences solution properties.

Abstract

We study the dependence of least nontrivial critical levels of the energy functional corresponding to the zero Dirichlet problem $-\Delta_p u = f(u)$ in a bounded domain $\Omega \subset \mathbb{R}^N$ upon domain perturbations. Assuming that the nonlinearity $f$ is superlinear and subcritical, we establish Hadamard-type formulas for such critical levels. As an application, we show that among all (generally eccentric) spherical annuli $\Omega$ least nontrivial critical levels attain maximum if and only if $\Omega$ is concentric. As a consequence of this fact, we prove the nonradiality of least energy nodal solutions whenever $\Omega$ is a ball or concentric annulus.

Equations

$$\{\begin{array}{rlrl}-{\Delta }_{p}u& =f(u)& & \text{in }\Omega ,\\ u& =0& & \text{on }\partial \Omega ,\end{array}$$

$${\lambda }_p(\Omega ):=\underset{u\in \text{accentset}\circ W_p^1(\Omega )\setminus \{0\}}{\min }\frac{{\int }_{\Omega }|\nabla u{|}^{p}dx}{{\int }_{\Omega }|u{|}^{p}dx}$$

$$F(s):={\int }_0^{s}f(t)dt.$$

$$E[u]=\frac{1}{p}{\int }_{\Omega }|\nabla u{|}^{p}dx-{\int }_{\Omega }F(u)dx.$$

$${\mu }_{+}(\Omega )=\underset{v\in \mathcal{N}(\Omega ),v⩾0}{\min }E[v]\text{and}{\mu }_{-}(\Omega )=\underset{v\in \mathcal{N}(\Omega ),v⩽0}{\min }E[v],$$

$$\mathcal{N}(\Omega ):=\{u\in \text{accentset}\circ W_p^1(\Omega )\setminus \{0\}:E^{\prime }[u]u\equiv {\int }_{\Omega }|\nabla u{|}^{p}dx-{\int }_{\Omega }uf(u)dx=0\}$$

$${\Phi }_t(x)=x+tR(x),R\in C^1({\mathbb{R}}^N,{\mathbb{R}}^N),|t|<\delta ,$$

$${\frac{\partial E[\alpha (v_t)v_t]}{\partial t}|}_{t=0}$$

$${\frac{\partial E[\alpha (w_t)w_t]}{\partial t}|}_{t=0}$$

$${\frac{\partial {\lambda }_p({\Omega }_t)}{\partial t}|}_{t=0}=-(p-1){\int }_{\partial \Omega }{\left|\frac{\partial {\phi }_p}{\partial n}\right|}^p⟨R,n⟩d\sigma ,$$

$${\mu }_q(\Omega )=\underset{u\in \text{accentset}\circ W_p^1(\Omega )\setminus \{0\}}{\min }J(u):=\underset{u\in \text{accentset}\circ W_p^1(\Omega )\setminus \{0\}}{\min }\frac{{\int }_{\Omega }|\nabla u{|}^{p}dx}{{\left({\int }_{\Omega }|u{|}^{q}dx\right)}^{\frac{p}{q}}}.$$

$${\frac{\partial J(u_t)}{\partial t}|}_{t=0}=-(p-1){\int }_{\partial \Omega }{\left|\frac{\partial u_0}{\partial n}\right|}^p⟨R,n⟩d\sigma .$$

$${\tilde{\mu }}_{\pm }(s)$$

$$\mathcal{M}:=\{u\in \text{accentset}\circ W_p^1(\Omega ):u^{+}\in \mathcal{N}(\Omega ),\text{leavevmode}-u^{-}\in \mathcal{N}(\Omega )\}.$$

$$\nu =\underset{u\in \mathcal{M}}{\min }E[u].$$

$${\Phi }_t(x)=x+tR(x),R\in C^1({\mathbb{R}}^N,{\mathbb{R}}^N),|t|<\delta ,$$

$$\frac{1}{p}{\int }_{\Omega }|\nabla u{|}^p\mathrm{div}(R)dx-{\int }_{\Omega }|\nabla u{|}^{p-2}⟨\nabla u,\nabla u\cdot R^{\prime }⟩dx$$

$$=-\frac{p-1}{p}{\int }_{\partial \Omega }{\left|\frac{\partial u}{\partial n}\right|}^p⟨R,n⟩d\sigma ,$$

$${\frac{\partial }{\partial t}{\int }_{{\Omega }_t}F(\varphi (t)u_t(y))dy|}_{t=0}={\varphi }^{\prime }(0){\int }_{\Omega }{u}_{0}f(\varphi (0)u_0)dx+{\int }_{\Omega }F(\varphi (0)u_0)\mathrm{div}(R)dx.$$

$${\int }_{{\Omega }_t}F(\varphi (t)u_t(y))dy={\int }_{\Omega }F(\varphi (t)u_t({\Phi }_t(x)))\det \left(\frac{d{\Phi }_t}{dx}\right)dx={\int }_{\Omega }F(\varphi (t)u_0)\det \left(I+t{R}^{\prime }\right)dx,$$

$${\frac{\partial }{\partial t}\det \left(I+t{R}^{\prime }\right)|}_{t=0}=\mathrm{Tr}(R^{\prime })=\mathrm{div}(R).$$

$${\frac{\partial }{\partial t}{\int }_{{\Omega }_t}|\nabla (\varphi (t)u_t(y)){|}^{p}dy|}_{t=0}$$

$$+|\varphi (0){|}^p{\int }_{\Omega }|\nabla u_0{|}^p\mathrm{div}(R)dx$$

$${\int }_{{\Omega }_t}|\nabla u_t(y){|}^{p}dy={\int }_{{\Omega }_t}|\nabla u_0({\Phi }_t^{-1}(y))\cdot ({\Phi }_t^{-1}(y){)}^{\prime }{|}^{p}dy={\int }_{\Omega }|\nabla u_0\cdot ({\Phi }_t^{\prime }{)}^{-1}{|}^p\det \left(I+t{R}^{\prime }\right)dx,$$

$${\frac{\partial }{\partial t}({\Phi }_t^{\prime }{)}^{-1}|}_{t=0}=-{({\Phi }_t^{\prime }{)}^{-1}\cdot \frac{\partial {\Phi }_t^{\prime }}{\partial t}\cdot ({\Phi }_t^{\prime }{)}^{-1}|}_{t=0}=-R^{\prime }$$

$${\frac{\partial }{\partial t}{\int }_{{\Omega }_t}|\nabla u_t(y){|}^{p}dy|}_{t=0}={\int }_{\Omega }|\nabla u_0{|}^p\mathrm{div}(R)dx-p{\int }_{\Omega }|\nabla u_0{|}^{p-2}⟨\nabla u_0,\nabla u_0\cdot R^{\prime }⟩dx.$$

$${\mu }_{+}({\Omega }_t)=\underset{v\in \mathcal{N}({\Omega }_t),v⩾0}{\min }E[v]\text{and}{\mu }_{-}({\Omega }_t)=\underset{v\in \mathcal{N}({\Omega }_t),v⩽0}{\min }E[v].$$

$$\Psi (\alpha ,t)={\alpha }^{p-1}{\int }_{{\Omega }_t}|\nabla v_t{|}^{p}dy-{\int }_{{\Omega }_t}{v}_{t}f(\alpha v_t)dy.$$

$${\Psi }_{\alpha }^{\prime }(1,0)=(p-1){\int }_{\Omega }|\nabla v_0{|}^{p}dx-{\int }_{\Omega }{v}_0^2{f}^{\prime }(v_0)dx={\int }_{\Omega }{v}_0^2\left((p-1)\frac{f(v_0)}{v_0}-f^{\prime }(v_0)\right)dx<0.$$

$${\frac{\partial E[{\alpha }_t{v}_t]}{\partial t}|}_{t=0}=-\frac{p-1}{p}{\int }_{\partial \Omega }{\left|\frac{\partial v_0}{\partial n}\right|}^p⟨R,n⟩d\sigma .$$

$${\frac{\partial E[{\alpha }_t{v}_t]}{\partial t}|}_{t=0}$$

$$-{\alpha }_0^{\prime }{\int }_{\Omega }{v}_{0}f(v_0)dx-{\int }_{\Omega }F(v_0)\mathrm{div}(R)dx.$$

Full text

Keywords: $p$-Laplacian; superlinear nonlinearity; domain derivative; shape optimization; Hadamard formula; Nehari manifold; least energy solution; nodal solution; nonradiality.

**MSC2010: 35J92, 35B06, 49Q10, 35B30, 49K30, 35B51. **

1 Introduction and main results

Let $\Omega$ be a bounded domain in $\mathbb{R}^{N}$, $N\geqslant 2$, with the boundary $\partial\Omega$ of class $C^{2,\varsigma}$, $\varsigma\in(0,1)$. Consider the boundary value problem

[TABLE]

where $\Delta_{p}u:=\text{div}(|\nabla u|^{p-2}\nabla u)$ is the $p$-Laplacian, $p>1$. Denote $p^{*}=\frac{Np}{N-p}$ if $p<N$ and $p^{*}=+\infty$ if $p\geqslant N$. We will always impose the following assumptions on the nonlinearity $f:\mathbb{R}\to\mathbb{R}$:

• $(A_{1})$

$f\in C^{1}(\mathbb{R}\setminus\{0\})\cap C^{0,\gamma}_{\text{loc}}(\mathbb{R})$ for some $\gamma\in(0,1)$.

• $(A_{2})$

There exist $q\in(p,p^{*})$ and $C>0$ such that $|sf^{\prime}(s)|,|f(s)|\leqslant C(|s|^{q-1}+1)$ for all $s\in\mathbb{R}\setminus\{0\}$.111If $p\geqslant N$, this assumption can be relaxed, see [33, condition (F4) and Lemma 5.6].

• $(A_{3})$

$f^{\prime}(s)>(p-1)\dfrac{f(s)}{s}>0$ for all $s\in\mathbb{R}\setminus\{0\}$, and $\limsup\limits_{s\to 0}\dfrac{f(s)}{|s|^{p-2}s}<\lambda_{p}(\Omega)$, where

[TABLE]

is the first eigenvalue of the zero Dirichlet $p$-Laplacian in $\Omega$.

• $(A_{4})$

There exist $s_{0}>0$ and $\theta>p$ such that $0<\theta F(s)\leqslant sf(s)$ for all $|s|>s_{0}$, where

[TABLE]

Problem ($\mathcal{D}$) corresponds to the energy functional $E:\accentset{\circ}{W}_{p}^{1}(\Omega)\to\mathbb{R}$ defined as

[TABLE]

The functional $E$ is weakly lower semicontinuous and belongs to $C^{1}(\accentset{\circ}{W}_{p}^{1}(\Omega))$. By definition, a weak solution of ($\mathcal{D}$) is a critical point of $E$. Moreover, any weak solution of ($\mathcal{D}$) is $C^{1,\beta}(\overline{\Omega})$-smooth, $\beta\in(0,1)$, and any constant-sign weak solution satisfies the Hopf maximum principle222see Remarks A.6 and A.7 in Appendix A..

If for some $c\in\mathbb{R}$ there exists a nontrivial critical point $u$ of $E$ such that $E[u]=c$, then $c$ is called a nontrivial critical level of $E$. We are interested in least nontrivial critical levels $\mu_{+}(\Omega)$ and $\mu_{-}(\Omega)$ among positive and negative solutions of ($\mathcal{D}$), respectively. In Appendix A below we discuss that under assumptions $(A_{1})-(A_{4})$, $\mu_{+}(\Omega)$ and $\mu_{-}(\Omega)$ can be defined as

[TABLE]

where

[TABLE]

is the Nehari manifold. Minimizers of (1.2) exist and they are least energy constant-sign solutions of ($\mathcal{D}$). Moreover, $\mu_{\pm}(\Omega)>0$.

The first goal of the present article is to study the behavior of $\mu_{\pm}(\Omega)$ under smooth domain perturbations $\Omega_{t}:=\Phi_{t}(\Omega)$ driven by a family of diffeomorphisms

[TABLE]

where $\delta>0$ is small enough. Let us take an arbitrary minimizer $v_{0}$ of $\mu_{+}(\Omega)$ and consider a function $v_{t}(y):=v_{0}(\Phi_{t}^{-1}(y))$, $y\in\Omega_{t}$. It is not hard to see that $v_{t}\in\accentset{\circ}{W}_{p}^{1}(\Omega_{t})$ and $v_{t}\geqslant 0$ on $\Omega_{t}$. By Lemma A.1 and Remark A.9 from Appendix A, for each $|t|<\delta$ we can find a unique constant $\alpha(v_{t})>0$ such that $\alpha(v_{t})v_{t}\in\mathcal{N}(\Omega_{t})$. Consequently, $\mu_{+}(\Omega_{t})\leqslant E[\alpha(v_{t})v_{t}]$. (We always assume by default that domains of integration in $E[\alpha(v_{t})v_{t}]$ are $\Omega_{t}$.) Analogous facts remain valid if we take any minimizer $w_{0}$ of $\mu_{-}(\Omega)$ and consider $w_{t}(y):=w_{0}(\Phi_{t}^{-1}(y))$, $y\in\Omega_{t}$.

We prove the following result.

Theorem 1.1**.**

Assume that $(A_{1})-(A_{4})$ are satisfied. Then $\mu_{+}(\Omega_{t})$ and $\mu_{-}(\Omega_{t})$ are continuous at $t=0$. Moreover, $E[\alpha(v_{t})v_{t}]$ and $E[\alpha(w_{t})w_{t}]$ are differentiable with respect to $t\in(-\delta,\delta)$ and the following Hadamard-type formulas hold:

[TABLE]

where $n$ is the outward unit normal vector to $\partial\Omega$ and $\left<\cdot,\cdot\right>$ stands for the scalar product in $\mathbb{R}^{N}$.

Remark 1.2**.**

It is rather counterintuitive that the domain derivative does not explicitly depend on the weak term $f$.

Origins of this problematic go back to the work of Hadamard [22], where he proved that the first eigenvalue $\lambda_{2}(\Omega_{t})$ of the zero Dirichlet Laplace operator in $\Omega_{t}$ is differentiable at $t=0$ and deduced its expression (see (1.5) below with $p=2$) which nowadays is known as the Hadamard formula. We refer the reader, for instance, to [36, 24, 16] for the general theory of the shape optimization and related historical remarks. The first eigenvalue (1.1) in the general case $p>1$ was treated in [19] (see also [29]), and it was proved that

[TABLE]

where $\varphi_{p}$ is the eigenfunction associated with $\lambda_{p}(\Omega)$ and normalized such that $\|\varphi_{p}\|_{L^{p}(\Omega)}=1$. At the same time, in Remark 3.5 below we discuss that $\mu_{+}(\Omega_{t})$ and $\mu_{-}(\Omega_{t})$ are not differentiable at $t=0$, in general.

Note that the main prototypical nonlinearity for ($\mathcal{D}$) is given by $f(u)=|u|^{q-2}u$, where $q\in(p,p^{*})$. It can easily be checked that assumptions $(A_{1})-(A_{4})$ are satisfied. Due to the homogeneity and oddness of $f$, the problem of finding the least critical levels $\mu_{\pm}(\Omega)$ can be rewritten in the form of the nonlinear Rayleigh quotient

[TABLE]

The minimum is achieved, and, after an appropriate normalization, corresponding minimizers satisfy ($\mathcal{D}$). These facts remain valid for all $q\in[1,p^{*})$.

As a corollary of the proof of Theorem 1.1 we obtain the following fact.

Theorem 1.3**.**

Let $q\in[1,p^{*})$. Then $\mu_{q}(\Omega_{t})$ is continuous at $t=0$. Moreover, if $u_{0}$ is a minimizer of $\mu_{q}(\Omega)$ normalized such that $\|u_{0}\|_{L^{q}(\Omega)}=1$, and $u_{t}(y):=u_{0}(\Phi_{t}^{-1}(y))$, $y\in\Omega_{t}$, then $J(u_{t})$ is differentiable with respect to $t\in(-\delta,\delta)$ and

[TABLE]

The second aim of our work is to use Theorem 1.1 for studying a shape optimization problem for $\mu_{\pm}(\Omega)$ over a special class of domains. Namely, let $\Omega$ be an open spherical annulus $B_{R_{1}}(x)\setminus\overline{B_{R_{0}}(y)}$, where $|x-y|<R_{1}-R_{0}$. Due to the invariance of ($\mathcal{D}$) upon orthogonal transformations of coordinates, we can take $x=0$ and $y=se_{1}$, where $s\in[0,R_{1}-R_{0})$ and $e_{1}$ is the first coordinate vector. For simplicity and to avoid ambiguity, we denote

[TABLE]

In order to guarantee the existence of minimizers of $\tilde{\mu}_{\pm}(s)$ for all $s\in[0,R_{1}-R_{0})$ (see Appendix A) the second part of assumption $(A_{3})$ must be satisfied for any annulus $B_{R_{1}}(0)\setminus\overline{B_{R_{0}}(se_{1})}$, $s\in[0,R_{1}-R_{0})$. To this end we impose the following additional assumption (see a discussion below):

• $(A_{3}^{*})$

$\limsup\limits_{s\to 0}\dfrac{f(s)}{|s|^{p-2}s}<\lambda_{p}(B_{R_{1}}(0)\setminus\overline{B_{R_{0}}((R_{1}-R_{0})e_{1})})$.

We consider the following question:

Which value of the displacement $s\in[0,R_{1}-R_{0})$ maximizes/minimizes $\tilde{\mu}_{\pm}(s)$?

In the case of the first eigenvalue (1.1) this question was addressed in several articles, see [25, 34, 23, 27] for the linear case $p=2$, and [12, 3] for general $p>1$. It was proved in [3, Theorem 1.1] that $\lambda_{p}(s):=\lambda_{p}(B_{R_{1}}(0)\setminus\overline{B_{R_{0}}(se_{1})})$ is continuous and strictly decreasing on $[0,R_{1}-R_{0})$, which implies that $\lambda_{p}(s)$ attains its maximum if and only if $s=0$ and attains its minimum if and only if $s=R_{1}-R_{0}$. These facts justify the choice $s=R_{1}-R_{0}$ in assumption $(A_{3}^{*})$.

The common approach to prove the monotonicity of $\lambda_{p}(s)$ is to consider a perturbation $\Phi_{t}$ which shifts the inner boundary $\partial B_{R_{0}}(se_{1})$ along the direction $e_{1}$ while the outer boundary $\partial B_{R_{1}}(0)$ remains fixed. Then the Hadamard formula (1.5) allows to find a derivative of $\lambda_{p}(s)$ with respect to the displacement $s$ in terms of an integral over the inner boundary. Hence, to show that $\lambda_{p}^{\prime}(s)<0$, one can try to compare values of the normal derivatives of the eigenfunction of $\lambda_{p}(s)$ on hemispheres $\{x\in\partial B_{R_{0}}(se_{1}):\,x_{1}<s\}$ and $\{x\in\partial B_{R_{0}}(se_{1}):\,x_{1}>s\}$. In the linear case $p=2$, reflection arguments together with the strong comparison principle can be applied to show that such values are strictly ordered, which leads to $\lambda_{p}^{\prime}(s)<0$ for any $s\in(0,R_{1}-R_{0})$. (Note that $\lambda_{p}^{\prime}(0)=0$ due to symmetry reasons.) At the same time, the lack of strong comparison principles for the general nonlinear case $p>1$ entails the use of additional arguments. In [12], applying an appropriate version of the weak comparison principle, it was shown that $\lambda_{p}^{\prime}(s)\leqslant 0$ for all $s\in(0,R_{1}-R_{0})$. The strict negativity of $\lambda_{p}^{\prime}(s)$ was obtained recently in [3] bypassing the usage of (global) strong comparison results.

Considering the least nontrivial critical levels $\tilde{\mu}_{\pm}(s)$, we follow the strategy described above. To this end, we employ two symmetrization methods: polarization [8, 7] and spherical symmetrization (i.e., foliated Schwarz symmetrization) [26, 7]. With the help of these methods, we use the ideas from [3], to derive the following result.

Theorem 1.4**.**

Assume that $(A_{1})-(A_{4})$ and $(A_{3}^{*})$ are satisfied. Then $\tilde{\mu}_{+}(s)$ and $\tilde{\mu}_{-}(s)$ are continuous and strictly decreasing on $[0,R_{1}-R_{0})$.

As a corollary of the proof of Theorem 1.4 we have the following fact which will be used later.

Proposition 1.5**.**

Assume that $(A_{1})-(A_{4})$ are satisfied. Then $\tilde{\mu}_{+}(s)$ and $\tilde{\mu}_{-}(s)$ are continuous and strictly decreasing for sufficiently small $s\geqslant 0$.

The last (but not least) aim of the present article is the investigation of symmetry properties of least energy nodal solutions to problem ($\mathcal{D}$) via the results stated above. By a nodal (or, equivalently, sign-changing) solution of ($\mathcal{D}$) we mean a weak solution $u$ such that $u^{\pm}:=\max\{\pm u,0\}\not\equiv 0$ in $\Omega$. By definition, a nodal set of $u$ is a set $Z=\overline{\{x\in\Omega:u(x)=0\}}$, and any connected component of $\Omega\setminus Z$ is a nodal domain of $u$.

Consider the nodal Nehari set

[TABLE]

Evidently, $\mathcal{M}$ contains all nodal solutions of ($\mathcal{D}$). Moreover, in Appendix A we discuss that a least energy nodal solution of ($\mathcal{D}$) can be found as a minimizer of the problem

[TABLE]

Let $\Omega$ be a bounded radial domain in $\mathbb{R}^{N}$, that is, $\Omega$ is a ball or concentric annulus. The study of symmetric properties of least energy nodal solutions to ($\mathcal{D}$) in such $\Omega$ was initiated in [7], where it was shown that in the linear case $p=2$ any minimizer of $\nu$ is a foliated Schwartz symmetric function with precisely two nodal domains. Here we consider the following question:

Is it true that any least energy nodal solution of ($\mathcal{D}$) in a bounded radial $\Omega$ is nonradial?

This question was first stated and answered affirmatively in [2] for the linear case $p=2$. The authors obtained the lower estimate $N+1$ on the Morse index of radial nodal solutions to ($\mathcal{D}$) and used the fact that the Morse index of any least energy nodal solution of ($\mathcal{D}$) is exactly $2$ (see [6]). Note that the assumption $E\in C^{2}(\accentset{\circ}{W}_{2}^{1}(\Omega))$ is essential for the arguments of [2] and [6]. Later, under weaker assumptions on $f$ which allow $E$ to be only $C^{1}(\accentset{\circ}{W}_{2}^{1}(\Omega))$, the nonradiality was proved in [5] by performing the idea of [2] in terms of a “generalized” Morse index. Nevertheless, necessity to work with the linearized problem associated with ($\mathcal{D}$) at sign-changing solutions makes a generalization of the methods of [2] and [5] to the case $p>1$ nonobvious. (See [9] about the linearization of the $p$-Laplacian). We also refer to [7, 37, 10] for some partial results on the nonradiality problem.

Here we give the affirmative answer on the above question on the nonradiality in the general case $p>1$ using different arguments based on shape optimization techniques.

Theorem 1.6**.**

Let $\Omega$ be a ball or annulus and let $(A_{1})-(A_{4})$ be satisfied. Then any minimizer of $\nu$ is nonradial and has precisely two nodal domains.

The fact that any minimizer of $\nu$ has exactly two nodal domains can be easily obtained by generalizing arguments from [11, p. 1051] or, equivalently, [6, p. 6]. To prove the nonradiality, we argue by contradiction and apply Proposition 1.5 to the least critical levels on eccentric annuli generated by small shifts of the nodal set of a radial nodal solution. Then, the union of least energy constant-sign solutions on modified in such a way nodal domains defines a function from $\mathcal{M}$ whose energy is strictly smaller than $\nu$.

To the best of our knowledge, the idea to use shape optimization techniques for studying properties of nodal solutions was firstly performed in [4], where it was proved that any second eigenfunction of the $p$-Laplacian on a ball cannot be radial. See also [3] about a development of this result.

It is worth mentioning that our approach has an intrinsic similarity with the methods of [2] and [5]. Consider a radial nodal solution $u$ of ($\mathcal{D}$) with $k$ nodal domains $D_{1},\dots,D_{k}$. Its nodal set is the union of $k-1$ concentric spheres $S_{1},\dots,S_{k-1}$ inside $\Omega$. According to Proposition 1.5, for any fixed $i\in\{1,\dots,k-1\}$ and $j\in\{1,\dots,N\}$, small shifts of $S_{i}$ along the coordinate axis $e_{j}$ generate a family of functions along which the energy functional $E$ strictly decreases. Thus, we have $(k-1)N$ such families generated by shifts. Moreover, considering the scaling $\alpha u|_{D_{i}}$, $\alpha>0$, for each of $k$ nodal components $D_{1},\dots,D_{k}$, we produce $k$ additional families of functions with strictly decreasing energy, see Lemma A.1. Therefore, in total, we have $(k-1)N+k$ such families (compare with [7, Theorem 2.2]). Without rigorous justification, we mention that this number can be seen as a weak variant of the Morse index of a radial nodal solution of ($\mathcal{D}$) with $k$ nodal domains.

The rest of the article is organized as follows. In Section 2, we study the dependence of $\mu_{\pm}(\Omega_{t})$ on $t$ and prove Theorems 1.1 and 1.3. Section 3 is devoted to the study of the shape optimization problem for annular domains and contains the proof of Theorem 1.4. In Section 4, we prove the nonradiality of least energy nodal solutions to ($\mathcal{D}$) stated in Theorem 1.6. Appendix A contains auxiliary results.

2 Domain perturbations for least nontrivial critical levels

For the proof of Theorems 1.1 and 1.3 we need to prepare several auxiliary facts. Recall that $\Omega_{t}=\Phi_{t}(\Omega)$ is the deformation of $\Omega$, where the diffeomorphism $\Phi_{t}$ is given by (1.3):

[TABLE]

and $\delta>0$ is sufficiently small.

Noting that any weak solution of ($\mathcal{D}$) in $\Omega$ belongs to $C^{1,\beta}(\overline{\Omega})$ (see Remark A.6), we state the following partial case of the generalized Pohozaev identity (see [15, Lemma 2, p. 323] with $\mathcal{L}(x,s,\xi)=\frac{1}{p}|\xi|^{p}-F(s)$).

Proposition 2.1**.**

Let $u$ be any weak solution of ($\mathcal{D}$) in $\Omega$. Then $u$ satisfies

[TABLE]

where $n$ is the outward unit normal vector to $\partial\Omega$, and $R^{\prime}$ is the Jacobi matrix of $R$.

Fix now a nontrivial $u_{0}\in\accentset{\circ}{W}_{p}^{1}(\Omega)$ and let $u_{t}\in\accentset{\circ}{W}_{p}^{1}(\Omega_{t})$ be a function defined as $u_{t}(y):=u_{0}(\Phi_{t}^{-1}(y))$, $y\in\Omega_{t}$. Although assertions of the following two lemmas can be deduced from general results [36, 24, 16], we give sketches of their proofs for the sake of completeness.

Lemma 2.2**.**

Let $\phi\in C^{1}(-\delta,\delta)$. Then $\int_{\Omega_{t}}F(\phi(t)u_{t}(y))\,dy$ is differentiable with respect to $t\in(-\delta,\delta)$ and

[TABLE]

Proof.

Changing variables by the rule $y=\Phi_{t}(x)$ and noting that $dy=\,{\rm det}\left(\frac{d\Phi_{t}}{dx}\right)dx$ for $|t|<\delta$, we obtain that

[TABLE]

where $R^{\prime}$ is the Jacobi matrix of $R$. This implies the differentiability of $\int_{\Omega_{t}}F(\phi(t)u_{t}(y))\,dy$ on $(-\delta,\delta)$. On the other hand, from Jacobi’s formula we know that

[TABLE]

Thus, differentiating $\int_{\Omega_{t}}F(\phi(t)u_{t}(y))\,dy$ by $t$ at zero, we derive (2.2). ∎

By the same arguments as in the proof of Lemma 2.2 we get the following fact.

Corollary 2.3**.**

$\int_{\Omega_{t}}u_{t}(y)f(\alpha u_{t}(y))\,dy$* is differentiable with respect to $t\in(-\delta,\delta)$ for any $\alpha\in\mathbb{R}$.*

Lemma 2.4**.**

Let $\phi\in C^{1}(-\delta,\delta)$. Then $\int_{\Omega_{t}}|\nabla(\phi(t)u_{t}(y))|^{p}\,dy$ is differentiable with respect to $t\in(-\delta,\delta)$ and

[TABLE]

Proof.

First, after the same change of variables as in the proof of Lemma 2.2, we obtain

[TABLE]

where by $(\Phi^{-1}_{t})^{\prime}$ and $\Phi_{t}^{\prime}$ we denote the corresponding Jacobi matrices and used the inversion property $(\Phi^{-1}_{t}(y))^{\prime}=(\Phi_{t}^{\prime}(x))^{-1}$. Hence, (2.5) implies the differentiability of $\int_{\Omega_{t}}|\nabla u_{t}(y)|^{p}\,dy$ on $(-\delta,\delta)$. Note that the derivative of the inverse Jacobi matrix $(\Phi_{t}^{\prime})^{-1}$ is given by

[TABLE]

since $\Phi^{\prime}_{t}=I$ for $t=0$. Hence, differentiating (2.5) by $t$ at zero and taking into account (2.3), we obtain

[TABLE]

Finally, noting that $\nabla(\phi(t)u_{t})=\phi(t)\nabla u_{t}$, we arrive at (2.4). ∎

Recall the definition (1.2) of the least nontrivial critical levels of $E$ in perturbed domains $\Omega_{t}$:

[TABLE]

From Appendix A (see Lemma A.3 and Remark A.9) we know that $\delta>0$ can be chosen sufficiently small such that $\mu_{+}(\Omega_{t})$ and $\mu_{-}(\Omega_{t})$ possess minimizers for any $|t|<\delta$ which are constant-sign $C^{1,\beta}(\overline{\Omega_{t}})$-solutions of ($\mathcal{D}$) in $\Omega_{t}$.

Below in this section, we always denote by $v_{0}$ an arbitrary minimizer of $\mu_{+}(\Omega)$, that is, $v_{0}\in\mathcal{N}(\Omega)$, $v_{0}\geqslant 0$ in $\Omega$, and $E[v_{0}]=\mu_{+}(\Omega)$. As above, consider the family of nonnegative functions $v_{t}(y):=v_{0}(\Phi_{t}^{-1}(y))$, where $y\in\Omega_{t}$ and $|t|<\delta$. We do not know that $v_{t}\in\mathcal{N}(\Omega_{t})$. However, for each $|t|<\delta$ Lemma A.1 yields the existence of a unique $\alpha_{t}=\alpha(v_{t})$ such that $\alpha_{t}>0$ and $\alpha_{t}v_{t}\in\mathcal{N}(\Omega_{t})$.

Lemma 2.5**.**

$\alpha_{t}\in C^{1}(-\delta,\delta)$* and $\alpha_{0}=1$.*

Proof.

Define $\Psi:(0,+\infty)\times(-\delta,\delta)\to\mathbb{R}$ by

[TABLE]

From Lemma 2.4 and Corollary 2.3 we see that $\Psi(\alpha,\cdot)$ is differentiable on $(-\delta,\delta)$ for any $\alpha>0$. On the other hand, we know that $v_{0}>0$ in $\Omega$ (see Remark A.7) and hence $v_{t}>0$ in $\Omega_{t}$. Thus, from $(A_{1})$ it follows that $v_{t}f(\alpha v_{t})$ is differentiable with respect to $\alpha>0$ for each $x\in\Omega$ and $|t|<\delta$. Therefore, using $(A_{2})$, we see that $\Psi(\cdot,t)\in C^{1}(0,+\infty)$ for any $|t|<\delta$.

Since $v_{0}\in\mathcal{N}(\Omega)$, we have $\Psi(1,0)=0$. Moreover, in view of the first part of $(A_{3})$ we have

[TABLE]

Hence, taking $\delta>0$ smaller (if necessary), the implicit function theorem assures the existence of a differentiable function $\alpha_{t}:(-\delta,\delta)\to(0,+\infty)$ such that $\alpha_{0}=1$ and $\Psi(\alpha_{t},t)=0$ for all $t\in(-\delta,\delta)$, that is, $\alpha_{t}v_{t}\in\mathcal{N}(\Omega_{t})$. ∎

Remark 2.6**.**

Consider any minimizer $w_{0}$ of $\mu_{-}(\Omega)$ and its deformation $w_{t}=w_{0}(\Phi_{t}^{-1}(y))$, $y\in\Omega_{t}$. Then the result of Lemma 2.5 remains valid for $\alpha_{t}=\alpha(w_{t})$ such that $\alpha_{t}w_{t}\in\mathcal{N}(\Omega_{t})$.

Now we are ready to prove Theorem 1.1. We give the proof of each statement separately.

Proposition 2.7**.**

$E[\alpha_{t}v_{t}]$* is differentiable with respect to $t\in(-\delta,\delta)$ and*

[TABLE]

Proof.

First, $E[\alpha_{t}v_{t}]$ is differentiable due to Lemmas 2.5, 2.2 and 2.4. Moreover, applying equalities (2.2) and (2.4) with $\phi(t)=\alpha_{t}$ and recalling that $\alpha_{0}=1$, we compute

[TABLE]

Since $v_{0}\in\mathcal{N}(\Omega)$, the terms containing $\alpha_{0}^{\prime}$ cancel out and we arrive at

[TABLE]

Finally, applying the Pohozaev identity (2.1), we derive (2.6). ∎

Remark 2.8**.**

Consider any minimizer $w_{0}$ of $\mu_{-}(\Omega)$ and its deformation $w_{t}=w_{0}(\Phi_{t}^{-1}(y))$, $y\in\Omega_{t}$. Arguing as in Proposition 2.7, we see that $E[\alpha(w_{t})w_{t}]$ is also differentiable with respect to $t\in(-\delta,\delta)$ and satisfies the Hadamard-type formula (1.4).

Proposition 2.9**.**

$\mu_{+}(\Omega_{t})$* and $\mu_{-}(\Omega_{t})$ are continuous at $t=0$.*

Proof.

We give the proof for $\mu_{+}(\Omega_{t})$ only. The case of $\mu_{-}(\Omega_{t})$ can be handled in much the same way. Let us show that

[TABLE]

Suppose, by contradiction, that the first inequality does not hold. Consider a minimizer $v_{0}$ of $\mu_{+}(\Omega)$ and its deformation $v_{t}(y):=v_{0}(\Phi_{t}^{-1}(y))$, $y\in\Omega_{t}$. We know that $E[\alpha(v_{t})v_{t}]\geqslant\mu_{+}(\Omega_{t})$. Moreover, $E[\alpha(v_{t})v_{t}]$ is continuous with respect to $t\in(-\delta,\delta)$, see Proposition 2.7. Therefore, we get

[TABLE]

which is impossible.

Suppose now, contrary to our claim, that $\mu_{+}(\Omega)>\liminf\limits_{t\to 0}\mu_{+}(\Omega_{t})$. Let $\{t_{k}\}_{k\in\mathbb{N}}$ be a sequence such that $\mu_{+}(\Omega)>\lim\limits_{k\to+\infty}\mu_{+}(\Omega_{t_{k}})$, and let $u_{k}\in\accentset{\circ}{W}_{p}^{1}(\Omega_{t_{k}})$ be a minimizer of $\mu_{+}(\Omega_{t_{k}})$, $k\in\mathbb{N}$. We want to show that $\{u_{k}\}_{k\in\mathbb{N}}$ converges, up to a subsequence, to a minimizer of $\mu_{+}(\Omega)$. This will give us a contradiction. Consider a smooth (nonempty) domain $\hat{\Omega}\subset\bigcap_{k\in\mathbb{N}}\Omega_{t_{k}}$. Extending each element of $\accentset{\circ}{W}_{p}^{1}(\hat{\Omega})$ outside of $\hat{\Omega}$ by zero, we see that $\mathcal{N}(\hat{\Omega})\subset\mathcal{N}(\Omega_{t_{k}})$ for any $k\in\mathbb{N}$. Taking any $\xi\in C_{0}^{\infty}(\hat{\Omega})$, we apply Lemma A.1 to find an appropriate multiplier $\alpha>0$ such that $\alpha\xi\in\mathcal{N}(\hat{\Omega})$, and hence $\mu_{+}(\Omega_{t_{k}})\leqslant E[\alpha\xi]$ for any $k\in\mathbb{N}$. This implies that all $\|\nabla u_{k}\|_{L^{p}(\Omega_{t_{k}})}$ are uniformly bounded from above. Indeed, using $(A_{2})$, $(A_{4})$, and the first part of $(A_{3})$, we get

[TABLE]

where $C_{1}>0$ is chosen sufficiently large to be independent of $k$. Therefore, supposing that $\|\nabla u_{k}\|_{L^{p}(\Omega_{t_{k}})}\to+\infty$ as $k\to+\infty$ and recalling that $u_{k}\in\mathcal{N}(\Omega_{t_{k}})$, we obtain a contradiction:

[TABLE]

since $\theta>p$.

Consider now a bounded domain $\widetilde{\Omega}\supset\bigcup_{k\in\mathbb{N}}\Omega_{t_{k}}$. Extending each $u_{k}$ by zero outside of $\Omega_{t_{k}}$, we get $u_{k}\in\accentset{\circ}{W}_{p}^{1}(\widetilde{\Omega})$ and $\|\nabla u_{k}\|_{L^{p}(\widetilde{\Omega})}=\|\nabla u_{k}\|_{L^{p}(\Omega_{t_{k}})}$ for all $k\in\mathbb{N}$. Therefore, the boundedness of $\{u_{k}\}_{k\in\mathbb{N}}$ in $\accentset{\circ}{W}_{p}^{1}(\widetilde{\Omega})$ implies the existence of $u\in\accentset{\circ}{W}_{p}^{1}(\widetilde{\Omega})$ such that, up to a subsequence, $u_{k}\to u$ weakly in $\accentset{\circ}{W}_{p}^{1}(\widetilde{\Omega})$ and strongly in $L^{q}(\widetilde{\Omega})$, $q\in(p,p^{*})$. Moreover, since $u_{k}\geqslant 0$ in $\Omega_{t_{k}}$ for all $k\in\mathbb{N}$, we get $u\geqslant 0$ a.e. in $\widetilde{\Omega}$.

In Remark A.9 below we show that the second part of $(A_{3})$ yields $\limsup\limits_{s\to 0}\dfrac{f(s)}{|s|^{p-2}s}<\widetilde{C}<\lambda_{p}(\Omega_{t_{k}})$ for some $\widetilde{C}>0$ and all $k$ large enough. Thus, due to the previous inequality and $(A_{2})$, we can find $\mu\in(0,\widetilde{C})$ and $C_{2}>0$ such that $|f(s)|\leqslant\mu|s|^{p-1}+C_{2}|s|^{q-1}$ for all $s\in\mathbb{R}$, where $q\in(p,p^{*})$. Therefore, we get

[TABLE]

for some $C_{3}>0$. If we suppose that $\|\nabla u_{k}\|_{L^{p}(\Omega_{t_{k}})}\to 0$ as $k\to+\infty$, then for sufficiently large $k$ we obtain a contradiction since $\mu<\widehat{C}<\lambda_{p}(\Omega_{t_{k}})$ and $q>p$. Thus, there exists $C_{4}>0$ such that $\|\nabla u_{k}\|_{L^{p}(\Omega_{t_{k}})}>C_{4}$ for any $k$ large enough. This implies that $\int_{\Omega_{t_{k}}}u_{k}f(u_{k})\,dx>C_{4}$ and hence $u\not\equiv 0$ a.e. in $\widetilde{\Omega}$.

Applying the fundamental lemma of calculus of variations, it is not hard to see that $u\equiv 0$ a.e. in $\mathbb{R}^{N}\setminus\Omega$. Since $\partial\Omega\in C^{2,\gamma}$, we conclude that $u\in\accentset{\circ}{W}_{p}^{1}(\Omega)$ (see, e.g., [1, Theorem 5.29]), and hence by the weak convergence we have

[TABLE]

Further, Lemma A.1 implies the existence of $\alpha(u)>0$ such that $\alpha(u)u\in\mathcal{N}(\Omega)$. Moreover, $E[\alpha u]$ achieves its unique maximum with respect to $\alpha>0$ at $\alpha(u)$. On the other hand, since each $u_{k}\in\mathcal{N}(\Omega_{t_{k}})$, we deduce from Lemma A.1 that a unique point of maximum of $E[\alpha u_{k}]$ with respect to $\alpha>0$ is achieved at $\alpha=1$. Therefore,

[TABLE]

Thus, recalling that $u\geqslant 0$ a.e. in $\Omega$, we get a contradiction to the definition of $\mu_{+}(\Omega)$, and hence $\mu_{+}(\Omega)\leqslant\liminf\limits_{t\to 0}\mu_{+}(\Omega_{t})$. This completes the proof. ∎

Remark 2.10**.**

Proposition 2.9 implies that from any sequence of minimizers $u_{k}$ of $\mu_{+}(\Omega_{t_{k}})$ (or $\mu_{-}(\Omega_{t_{k}})$), $k\in\mathbb{N}$, one can extract a subsequence which converges strongly in $W^{1}_{p}(\mathbb{R}^{N})$ to a minimizer of $\mu_{+}(\Omega)$ (or $\mu_{-}(\Omega)$). In view of possible nonuniqueness, the limit minimizer may depend on a sequence $\{t_{k}\}_{k\in\mathbb{N}}$.

Theorem 1.3 can be proved using the same arguments as for Theorem 1.1 (even without normalization by $\alpha_{t}$ in view of homogeneity of the functional $J$ in (1.6)).

3 Optimization problem in annuli

In this section we prove Theorem 1.4. Let us fix $R_{1}>R_{0}>0$ and $s\in[0,R_{1}-R_{0})$. Consider problem ($\mathcal{D}$) in the open spherical annulus $B_{R_{1}}(0)\setminus\overline{B_{R_{0}}(se_{1})}$:

[TABLE]

Recall the notation $\tilde{\mu}_{\pm}(s)=\mu_{\pm}(B_{R_{1}}(0)\setminus\overline{B_{R_{0}}(se_{1})})$ for the least nontrivial critical levels defined by (1.2) and consider a diffeomorphism $\Phi_{t}(x)=x+tR(x)$, $|t|<\delta$, with the vector field $R(x)=\varrho(x)e_{1}$, where $\varrho$ is a smooth function equal to zero in a neighborhood of $\partial B_{R_{1}}(0)$ and equal to one in a neighborhood of $\partial B_{R_{0}}(se_{1})$. It is not hard to see that

[TABLE]

Therefore $\mu_{\pm}(\Phi_{t}(B_{R_{1}}(0)\setminus\overline{B_{R_{0}}(se_{1})}))=\tilde{\mu}_{\pm}(s+t)$. This fact, together with Proposition 2.9, implies the first part of Theorem 1.4.

Lemma 3.1**.**

Let $(A_{1})-(A_{4})$ be satisfied. Then $\tilde{\mu}_{+}(s)$ and $\tilde{\mu}_{-}(s)$ are continuous for sufficiently small $s\geqslant 0$. If moreover $(A_{3}^{*})$ holds, then $\tilde{\mu}_{+}(s)$ and $\tilde{\mu}_{-}(s)$ are continuous on $[0,R_{1}-R_{0})$.

Recall that imposing $(A_{3}^{*})$ we can find minimizers of $\tilde{\mu}_{\pm}(s)$ for each $s\in[0,R_{1}-R_{0})$, see the discussion in Section 1. Without $(A_{3}^{*})$ we can guarantee the existence of minimizers only for sufficiently small $s\geqslant 0$.

For simplicity of exposition we will give the proof of the second part of Theorem 1.4 for $\tilde{\mu}_{+}(s)$ only. The case of $\tilde{\mu}_{-}(s)$ can be proved along the same lines. We will always assume that $(A_{1})-(A_{4})$ and $(A_{3}^{*})$ are satisfied.

Let $v$ be an arbitrary minimizer of $\tilde{\mu}_{+}(s)$, that is, $v$ is a least energy positive solution of (3.1). Recall that $v\in C^{1,\beta}(\overline{\Omega})$ and satisfies the Hopf maximum principle (see Remarks A.6 and A.7 below). Defining $v_{t}(y):=v(\Phi_{t}^{-1}(y))$, $y\in B_{R_{1}}(0)\setminus\overline{B_{R_{0}}((s+t)e_{1})}$, we have $\tilde{\mu}_{+}(s+t)\leqslant E[\alpha(v_{t})v_{t}]$, where $\alpha(v_{t})$ is given by Lemma A.1. Hence, noting that $\alpha(v)=1$, from Theorem 1.1 we obtain

[TABLE]

where $n_{1}=n_{1}(x)$ is the first component of the outward unit normal vector $n$ to $\partial B_{R_{0}}(se_{1})$.

Our main aim is to prove that $D\tilde{\mu}_{+}(s)<0$ for all $s\in(0,R_{1}-R_{0})$. In combination with the continuity of $\tilde{\mu}_{+}(s)$ (see Lemma 3.1), this will immediately imply the desired strict monotonicity of $\tilde{\mu}_{+}(s)$ on $[0,R_{1}-R_{0})$.

For the fixed $s\in[0,R_{1}-R_{0})$ we write $\Omega:=B_{R_{1}}(0)\setminus\overline{B_{R_{0}}(se_{1})}$, for simplicity. Denote by $H_{a}$ a hyperplane passing through the point $se_{1}$ (center of the inner ball) perpendicularly to a vector $a\neq 0$ which satisfies $\left<a,e_{1}\right>\geqslant 0$. Let $\rho_{a}:\mathbb{R}^{N}\to\mathbb{R}^{N}$ be a map which reflects a point $x\in\mathbb{R}^{N}$ with respect to $H_{a}$, and $\Sigma_{a}:=\{x\in\mathbb{R}^{N}:\,\left<a,x-se_{1}\right>>0\}$. Note that under the assumption on $a$ we have $\rho_{a}(\Omega\cap\Sigma_{a})\subseteq\{x\in\Omega:\,\left<a,x-se_{1}\right><0\}$.

First we prove the following fact.

Lemma 3.2**.**

$D\tilde{\mu}_{+}(s)\leqslant 0$* for all $s\in[0,R_{1}-R_{0})$. Moreover, if $D\tilde{\mu}_{+}(s)=0$ for some $s\in[0,R_{1}-R_{0})$, then for any minimizer $v$ of $\tilde{\mu}_{+}(s)$ there exists $\varepsilon_{0}>0$ such that $v(x)=v(\rho_{e_{1}}(x))$ for all $x\in\partial B_{R_{0}+\varepsilon}(se_{1})$ and $\varepsilon\in(0,\varepsilon_{0})$.*

Proof.

Let $v$ be a minimizer of $\tilde{\mu}_{+}(s)$ for some $s\in[0,R_{1}-R_{0})$. Extend $v$ by zero outside of $\Omega$ and consider the following function:

[TABLE]

The function $V$ is the polarization of $v$ with respect to $H_{e_{1}}$, see, e.g., [8, 7]. It is known that $V\in\accentset{\circ}{W}_{p}^{1}(\Omega)$, $V\geqslant 0$ in $\Omega$, and

[TABLE]

see [8, Corollary 5.1] and [7, Lemma 2.2]. In particular, $V\in\mathcal{N}(\Omega)$ and $E[V]=E[v]$, that is, $V$ is also a minimizer of $\tilde{\mu}_{+}(s)$. Since (3.2) holds for an arbitrary minimizer of $\tilde{\mu}_{+}(s)$, we arrive at

[TABLE]

Now, since $V(x)=V(\rho_{e_{1}}(x))=0$ for all $x\in\partial B_{R_{0}}(se_{1})$, and $V(x)\leqslant V(\rho_{e_{1}}(x))$ for all $x\in\Sigma_{e_{1}}$, we get

[TABLE]

Moreover, noting that $n_{1}(x)=-n_{1}(\rho_{e_{1}}(x))$ and $n_{1}(x)<0$ for all $x\in\partial B_{R_{0}}(se_{1})\cap\Sigma_{e_{1}}$, we get

[TABLE]

This is the desired conclusion.

Let us prove the second part of the lemma. Suppose that $D\tilde{\mu}_{+}(s)=0$ for some $s\in[0,R_{1}-R_{0})$. Polarizing any minimizer $v$ of $\tilde{\mu}_{+}(s)$ as above, we conclude from (3.4) and (3.5) that

[TABLE]

Define a function $w(x)=V(\rho_{e_{1}}(x))-V(x)$. By the properties of $V$ we have $w\geqslant 0$ in $\Omega\cap\Sigma_{e_{1}}$ and $w=0$ on $\partial B_{R_{0}}(se_{1})\cap\Sigma_{e_{1}}$. Moreover, since $\frac{\partial V}{\partial n}<0$ on $\partial B_{R_{0}}(se_{1})$, we can find $\varepsilon_{0}>0$ such that for any $\varepsilon\in(0,\varepsilon_{0})$ there exists $\eta>0$ such that $|\nabla V|>\eta$ in $B_{R_{0}+\varepsilon}(se_{1})\setminus B_{R_{0}}(se_{1})$. Therefore, using Remark A.8, we can linearize the difference $\Delta_{p}V(\rho_{e_{1}}(\cdot))-\Delta_{p}V$ in $(B_{R_{0}+\varepsilon}(se_{1})\setminus B_{R_{0}}(se_{1}))\cap\Sigma_{e_{1}}$ as in the proof of [18, Proposition 5.1] on p. 1239 and obtain that $w$ satisfies pointwise the following linear elliptic inequality in this set:

[TABLE]

(The inequality in (3.7) follows from the monotonicity of $f$ on $\mathbb{R}$, see $(A_{3})$.) The matrix $\{a_{ij}\}_{i,j=1}^{N}$ is symmetric and there exist $C_{1}=C_{1}(\eta)>0$ and $C_{2}=C_{2}(\eta)>0$ such that

[TABLE]

for any $x\in(B_{R_{0}+\varepsilon}(se_{1})\setminus B_{R_{0}}(se_{1}))\cap\Sigma_{e_{1}}$ and $\xi\in\mathbb{R}^{N}\setminus\{0\}$. That is, the elliptic operator on the left-hand side of (3.7) is uniformly elliptic. Moreover, each $b_{i}\in L^{\infty}((B_{R_{0}+\varepsilon}(se_{1})\setminus B_{R_{0}}(se_{1}))\cap\Sigma_{e_{1}})$. Hence, the classical strong maximum principle [20, Theorem 3.5, p. 35] implies that either $V(x)=V(\rho_{e_{1}}(x))$ for any $x\in(B_{R_{0}+\varepsilon}(se_{1})\setminus B_{R_{0}}(se_{1}))\cap\Sigma_{e_{1}}$, or $V(x)<V(\rho_{e_{1}}(x))$ in the same set and

[TABLE]

by [20, Lemma 3.4, p. 34]. However, in view of (3.6) only the first case can occur. Thus, from the definition of the polarization we obtain the desired fact: $v(x)=v(\rho_{e_{1}}(x))$ for any $x\in B_{R_{0}+\varepsilon}(se_{1})\setminus B_{R_{0}}(se_{1})$ and $\varepsilon\in(0,\varepsilon_{0})$. ∎

Remark 3.3**.**

In the second part of the proof of Lemma 3.2, $V$ and $V(\rho_{e_{1}}(\cdot))$ satisfy $V(x)<V(\rho_{e_{1}}(x))$ for all $x\in\partial B_{R_{1}}(0)\cap\Sigma_{e_{1}}$ whenever $s\in(0,R_{1}-R_{0})$. Therefore, in the case $p=2$ the classical strong maximum (comparison) principle implies that $V(x)<V(\rho_{e_{1}}(x))$ for all $x\in\Omega\cap\Sigma_{e_{1}}$ and (3.8) holds. This yields $D\tilde{\mu}_{+}(s)<0$ for any $s\in(0,R_{1}-R_{0})$. However, the lack of strong comparison principles in the general case $p>1$ does not allow to conclude directly that $D\tilde{\mu}_{+}(s)<0$. (We refer to [14, 35] for versions of the strong comparison principle under the restriction $p>\frac{2N+2}{N+2}$.) On the other hand, the arguments which we use below do not require any global strong comparison result and rely mainly on the (local) strong comparison principle near the boundary of $\Omega$, where the $p$-Laplacian is neither degenerate nor singular thanks to the Hopf maximum principle.

Now we show the following result on existence of axially symmetric minimizers of $\tilde{\mu}_{+}(s)$.

Lemma 3.4**.**

For any $s\in[0,R_{1}-R_{0})$ there exists a minimizer of $\tilde{\mu}_{+}(s)$ invariant under rotations around axis $e_{1}$.

Proof.

Let $v$ be an arbitrary minimizer of $\tilde{\mu}_{+}(s)$ for some $s\in[0,R_{1}-R_{0})$. Recall that $v\in C^{1,\beta}(\overline{\Omega})$ and $v>0$ in $\Omega$, see Remarks A.6 and A.7 below. To prove the assertion we apply the spherical symmetrization for $v$ with respect to $-e_{1}$. Namely, for a set $A\subset\mathbb{R}^{N}$ its spherical symmetrization around $-e_{1}$ is a set $A^{*}$ defined such that for any $r>0$, $A^{*}\cap\partial B_{r}(0)$ is a spherical cap of $\partial B_{r}(0)$ with the pole $-re_{1}$ and $\text{meas}(A^{*}\cap\partial B_{r}(0))=\text{meas}(A\cap\partial B_{r}(0))$, see, e.g., [26, 7]. Then, the spherical symmetrization of $v$ around $-e_{1}$ is a function $v^{*}:\mathbb{R}^{N}\to\mathbb{R}$ defined as

[TABLE]

By construction, $v^{*}$ is invariant under rotations around $e_{1}$. Due to the symmetry of $\Omega=B_{R_{1}}(0)\setminus\overline{B_{R_{0}}(se_{1})}$, we have $\Omega^{*}=\Omega$. Thus, $v^{*}\in\accentset{\circ}{W}_{p}^{1}(\Omega)$, as follows from [26, property (L), p. 20]. Moreover, [26, properties (G1), p. 26, and (C), p. 22] imply that

[TABLE]

If in the first expression equality holds, then $v^{*}\in\mathcal{N}(\Omega)$, $v^{*}\geqslant 0$ and $E[v^{*}]=E[v]$, that is, $v^{*}$ is a minimizer of $\tilde{\mu}_{+}(s)$ with the desired properties. Else, we get a contradiction. Indeed, using Lemma A.1, we can find $\alpha^{*}\in(0,1)$ such that $\alpha^{*}v^{*}\in\mathcal{N}(\Omega)$. However,

[TABLE]

where the strict inequality follows from the first part of $(A_{3})$, which is impossible. ∎

Remark 3.5**.**

In [13] it was proved that in the case $p=2$ and $N=2$, $\tilde{\mu}_{+}(0)$ possesses a nonradial minimizer $v$ if the annulus is sufficiently thin (see also [32, 33, 28] and references therein for the development of this result for $p>1$ and $N\geqslant 2$). Using Lemma 3.4, we can assume that $v$ is axially symmetric with respect to $e_{1}$. Moreover, the spherical symmetrization implies, in fact, that $v$ is polarized with respect to $H_{e_{1}}$, that is, $v(x)\leqslant v(\rho_{e_{1}}(x))$ for all $x\in\Omega\cap\Sigma_{e_{1}}$. However, since $v$ is nonradial, the classical strong maximum principle implies that $v(x)<v(\rho_{e_{1}}(x))$ in this domain, which yields $D\tilde{\mu}_{+}(0)<0$, see (3.5). This fact contradicts the possible differentiability of $\tilde{\mu}_{+}(s)$ at $s=0$. Indeed, if $\tilde{\mu}_{+}(0)$ is differentiable, then $D\tilde{\mu}_{+}(0)=(\tilde{\mu}_{+}(0))^{\prime}_{s}$ and we must have $(\tilde{\mu}_{+}(0))^{\prime}_{s}=0$, due to the symmetry of $\Omega$.

The following lemma provides the main ingredient for the proof of $D\tilde{\mu}_{+}(s)<0$ for $s\in(0,R_{1}-R_{0})$. (See [3, Theorem 3.8] about the analogous properties for the first eigenvalue $\lambda_{p}(s)$.)

Lemma 3.6**.**

Let $D\tilde{\mu}_{+}(s)=0$ for some $s\in[0,R_{1}-R_{0})$. Then for any axially symmetric (with respect to $e_{1}$) minimizer $v$ of $\tilde{\mu}_{+}(s)$ there is a ball $B_{r_{0}}(se_{1})$ with $r_{0}\in(R_{0},R_{1}-s)$ such that $v$ is radial in the annulus $B_{r_{0}}(se_{1})\setminus B_{R_{0}}(se_{1})$. Moreover, $|\nabla v|=0$ on $\partial B_{r_{0}}(se_{1})$ and $v\in C^{2}(B_{r_{0}}(se_{1})\setminus B_{R_{0}}(se_{1}))$.

Proof.

Let $D\tilde{\mu}_{+}(s)=0$ for some $s\in[0,R_{1}-R_{0})$ and let $v$ be a minimizer of $\tilde{\mu}_{+}(s)$ which is invariant under rotations around $e_{1}$ (see Lemma 3.4). Due to the Hopf maximum principle (see Remark A.7 below) we can find $\varepsilon_{0}>0$ (as in Lemma 3.2) such that for any $\varepsilon\in(0,\varepsilon_{0})$ there exists $\eta>0$ such that $|\nabla v|>\eta$ in $B_{R_{0}+\varepsilon}(se_{1})\setminus B_{R_{0}}(se_{1})$.

Suppose, contrary to the radiality of $v$, that for some $\varepsilon\in(0,\varepsilon_{0})$ there exist $\hat{x},\hat{y}\in\partial B_{R_{0}+\varepsilon}(se_{1})$ such that $v(\hat{x})\neq v(\hat{y})$. Note that $\hat{y}\neq\rho_{e_{i}}(\hat{x})$ for $i=2,\dots,N$ since $v$ is axially symmetric with respect to $e_{1}$. Moreover, $\hat{y}\neq\rho_{e_{1}}(\hat{x})$, as it follows from Lemma 3.2. Let $\bar{x},\bar{y}\in\partial B_{R_{0}+\varepsilon}(se_{1})$ lie on the opposite sides of the diameter of $B_{R_{0}+\varepsilon}(se_{1})$ which is collinear to $\hat{x}-\hat{y}$. Assume, without loss of generality, that $\bar{x}_{1}\geqslant s$. Using the symmetries of $v$ given by Lemmas 3.2 and 3.4, we derive that

[TABLE]

Let us denote $c=\bar{x}-\bar{y}$ and consider the polarization of $v$ with respect to $H_{c}$:

[TABLE]

Define a function $w(x)=V_{c}(\rho_{c}(x))-V_{c}(x)$. We have $w\geqslant 0$ in $\Omega\cap\Sigma_{c}$ and $w=0$ on $\partial B_{R_{0}}(se_{1})\cap\Sigma_{c}$. Using the linearization of the $p$-Laplacian as in the proof of Lemma 3.2, we see that $w$ satisfies

[TABLE]

pointwise in $(B_{R_{0}+\varepsilon}(se_{1})\setminus\overline{B_{R_{0}}(se_{1})})\cap\Sigma_{c}$. Hence, either $V_{c}(x)=V_{c}(\rho_{c}(x))$ for all $x\in(B_{R_{0}+\varepsilon}(se_{1})\setminus\overline{B_{R_{0}}(se_{1})})\cap\Sigma_{c}$, or $V_{c}(x)<V_{c}(\rho_{c}(x))$ for all $x\in(B_{R_{0}+\varepsilon}(se_{1})\setminus\overline{B_{R_{0}}(se_{1})})\cap\Sigma_{c}$. However, we have simultaneously $V_{c}(\hat{x})<V_{c}(\hat{y})\equiv V_{c}(\rho_{c}(\hat{x}))$ and $V_{c}(\bar{x})=V_{c}(\bar{y})\equiv V_{c}(\rho_{c}(\bar{x}))$, which is impossible.

Finally, considering the least $\varepsilon_{0}>0$ such that $|\nabla v(x)|=0$ occurs for some $x\in\partial B_{R_{0}+\varepsilon_{0}}(se_{1})$, we obtain that $v$ is radial in the annulus $B_{R_{0}+\varepsilon}(se_{1})\setminus B_{R_{0}}(se_{1})$ for any $\varepsilon\in(0,\varepsilon_{0})$. Denoting $r_{0}=R_{0}+\varepsilon_{0}$ and referring to Remark A.8 below for the $C^{2}$-regularity, we finish the proof of the lemma. ∎

Let us now outline how to prove symmetry results similar to Lemma 3.6, but in a neighborhood of the outer ball $B_{R_{1}}(0)$. Consider a diffeomorphism $\bar{\Phi}_{t}(x)=x+t\bar{R}(x)$, where the vector field $\bar{R}(x)=-\bar{\varrho}(x)e_{1}$ and $\bar{\varrho}$ is a smooth function equal to one in a neighborhood of $\partial B_{R_{1}}(0)$ and equal to zero in a neighborhood of $\partial B_{R_{0}}(se_{1})$. We see that

[TABLE]

for all $|t|$ small enough. Taking into account the invariance of $\mu_{+}(\Omega)$ under translations of $\Omega$, we get $\mu_{+}(\bar{\Phi}_{t}(B_{R_{1}}(0)\setminus\overline{B_{R_{0}}(se_{1})}))=\tilde{\mu}_{+}(s+t)$. Therefore, similarly to (3.2), we obtain the following upper estimate for $D\tilde{\mu}_{+}(s)$:

[TABLE]

Denote by $\bar{H}_{a}$ a hyperplane passing through the origin (center of the outer ball) perpendicularly to a vector $a\neq 0$ which satisfies $\left<a,e_{1}\right>\geqslant 0$. Let $\bar{\rho}_{a}(x)$ be a reflection of $x\in\mathbb{R}^{N}$ with respect to $\bar{H}_{a}$, and $\bar{\Sigma}_{a}=\{x\in\mathbb{R}^{N}:\,\left<a,x\right>>0\}$. Under the assumption on $a$, we have $\bar{\rho}_{a}(\Omega\cap\bar{\Sigma}_{a})\subseteq\{x\in\Omega:\,\left<a,z\right><0\}$. Consider the corresponding polarization of a minimizer $v$ of $\tilde{\mu}_{+}(s)$:

[TABLE]

It is not hard to see that $\text{supp}\,\bar{V}_{a}=\overline{\Omega}$ and hence $\bar{V}_{a}\in\accentset{\circ}{W}_{p}^{1}(\Omega)$.

Arguing now along the same lines as in the proofs of Lemmas 3.2 and 3.6 with the use of (3.10) instead of (3.2), polarizations $\bar{V}_{e_{1}}$ and $\bar{V}_{c}$ instead of (3.3) and (3.9), respectively, and the linearization of the $p$-Laplacian in a neighborhood of $\partial B_{R_{1}}(0)$ instead of $\partial B_{R_{0}}(se_{1})$, we obtain the following results.

Lemma 3.7**.**

Let $D\tilde{\mu}_{+}(s)=0$ for some $s\in[0,R_{1}-R_{0})$. Then for any minimizer $v$ of $\tilde{\mu}_{+}(s)$ there exists $\varepsilon_{1}>0$ such that $v(x)=v(\bar{\rho}_{e_{1}}(x))$ for all $x\in\partial B_{R_{1}-\varepsilon}(0)$ and $\varepsilon\in(0,\varepsilon_{1})$.

Lemma 3.8**.**

Let $D\tilde{\mu}_{+}(s)=0$ for some $s\in[0,R_{1}-R_{0})$. Then for any axially symmetric (with respect to $e_{1}$) minimizer $v$ of $\tilde{\mu}_{+}(s)$ there is a ball $B_{r_{1}}(0)$ with $r_{1}\in(R_{0}+s,R_{1})$ such that $v$ is radial in the annulus $B_{R_{1}}(0)\setminus B_{r_{1}}(0)$. Moreover, $|\nabla v|=0$ on $\partial B_{r_{1}}(0)$ and $v\in C^{2}(\overline{B_{R_{1}}(0)}\setminus\overline{B_{r_{1}}(0}))$.

Now we are ready to prove the main result which implies strict monotonicity of $\tilde{\mu}_{+}(s)$ on $[0,R_{1}-R_{0})$, that is, the second part of Theorem 1.4.

Proposition 3.9**.**

$D\tilde{\mu}_{+}(s)<0$* for all $s\in(0,R_{1}-R_{0})$.*

Proof.

Suppose, by contradiction, that $D\tilde{\mu}_{+}(s)=0$ for some $s\in(0,R_{1}-R_{0})$. Let $v$ be an axially symmetric minimizer of $\tilde{\mu}_{+}(s)$ given by Lemma 3.4. From Lemmas 3.6 and 3.8 we know that there exist $r_{0}$ and $r_{1}$ such that $v\in C^{2}(\overline{B_{R_{1}}(0)}\setminus B_{r_{1}+\varepsilon}(0))$ and $v\in C^{2}(\overline{B_{r_{0}-\varepsilon}(se_{1})}\setminus B_{R_{0}}(se_{1}))$ for any sufficiently small $\varepsilon>0$, and hence $v$ satisfies ($\mathcal{D}$) pointwise in the corresponding domains. Let us multiply ($\mathcal{D}$) by $v$, integrate it over $B_{R_{1}}(0)\setminus\overline{B_{r_{1}+\varepsilon}(0)}$ and tend $\varepsilon\to 0$. We get

[TABLE]

since $v=0$ on $\partial B_{R_{1}}(0)$ and $|\nabla v|\to 0$ on $\partial B_{r_{1}+\varepsilon}(0)$ as $\varepsilon\to 0$, due to Lemma 3.8. Analogously, we obtain

[TABLE]

Note that

[TABLE]

and similar decompositions hold for $\int_{\Omega}vf(v)\,dx$ and $\int_{\Omega}F(v)\,dx$. Therefore, recalling that $v\in\mathcal{N}(\Omega)$ and using (3.11) and (3.12), we derive also

[TABLE]

In other words, $v$ satisfies the Nehari constraint over each of the domains

[TABLE]

Let us consider a function $w:\Omega\to\mathbb{R}$ defined by

[TABLE]

where constants $C_{1},C_{2}>0$ are chosen such that $C_{1}v|_{\partial B_{r_{1}}(0)}=C_{2}=v|_{\partial B_{r_{0}}(se_{1})}$. (Note that $v$ is constant on $\partial B_{r_{1}}(0)$ and $\partial B_{r_{0}}(se_{1})$ due to Lemmas 3.6 and 3.8, respectively.) Therefore, $w>0$ in $\Omega$, $w\in C^{1}(\overline{\Omega})$, and there exists a unique $\alpha(w)>0$ such that $\alpha(w)w\in\mathcal{N}(\Omega)$ and $E[\alpha w]$ achieves a global maximum with respect to $\alpha>0$ at $\alpha(w)$, see Lemma A.1. Thus, we have

[TABLE]

where $E_{1}$ denotes a restriction of $E$ to the domain of integration $B_{R_{1}}(0)\setminus\overline{B_{r_{1}}(0)}$, etc.

Recalling (3.11) and (3.12), Lemma A.2 implies that $E_{1}[\alpha(w)C_{1}v]\leqslant E_{1}[v]$ and $E_{3}[\alpha(w)v]\leqslant E_{3}[v]$. Moreover, since $v>0$ in $\Omega$, from (3.13) we get $E_{2}[v]>0$ by Lemma A.2. However,

[TABLE]

Thus,

[TABLE]

and we get a contradiction with (3.14). ∎

4 Nonradiality of least energy nodal solutions

In this section we prove Theorem 1.6, that is, we show that any least energy nodal solution of problem ($\mathcal{D}$) in a ball or annulus is nonradial. First, we treat the case of a ball. Consider the problem

[TABLE]

where $B_{R}(0)$ is the open ball with some radius $R$ centered at the origin, and $f$ satisfies $(A_{1})-(A_{4})$. Recall that any least energy nodal solution of (4.1) is a minimizer of

[TABLE]

where $\mathcal{M}$ is the nodal Nehari set (1.7).

Suppose, by contradiction, that there exists a minimizer $u$ of $\nu$ which is radial. Hence, there exists $r\in(0,R)$ such that, without loss of generality, $u^{+}$ is a least energy positive solution of ($\mathcal{D}$) in the annulus $B_{R}(0)\setminus\overline{B_{r}(0)}$ and $-u^{-}$ is a least energy negative solution of ($\mathcal{D}$) in the ball $B_{r}(0)$. As in Section 3, let us perturb $B_{R}(0)\setminus\overline{B_{r}(0)}$ by shifting the inner ball in direction $e_{1}$. From Lemma A.3 and Remark A.9 we know that $\mu_{+}(B_{R}(0)\setminus\overline{B_{r}(se_{1})})$ possesses a minimizer $v_{s}$ for any $s\geqslant 0$ small enough, and

[TABLE]

by Proposition 1.5. Extending $v_{s}$ by zero outside of $B_{R}(0)\setminus\overline{B_{r}(se_{1})}$, we get $v_{s}\in\mathcal{N}(B_{R}(0)\setminus\overline{B_{r}(se_{1})})\subset\mathcal{N}(B_{R}(0))$. On the other hand, it is not hard to see that the translation $-u^{-}(\cdot-se_{1})\in\mathcal{N}(B_{R}(0))$ and $E[-u^{-}(\cdot-se_{1})]=E[-u^{-}]$. Therefore, if we consider a function $U_{s}$ defined as $U_{s}(x)=v_{s}(x)-u^{-}(x-se_{1})$, $x\in B_{R}(0)$, then $U_{s}\in\mathcal{M}$. But

[TABLE]

in view of (4.2), which is impossible.

Consider now problem ($\mathcal{D}$) in some annulus $B_{R_{1}}(0)\setminus\overline{B_{R_{0}}(0)}$. Suppose that this problem possesses a least energy nodal solution $u$ which is radial. Hence, there exists $r\in(R_{0},R_{1})$ such that, without loss of generality, $u^{+}$ is a least energy positive solution of ($\mathcal{D}$) in $B_{R_{1}}(0)\setminus\overline{B_{r}(0)}$ and $-u^{-}$ is a least energy negative solution of ($\mathcal{D}$) in $B_{r}(0)\setminus\overline{B_{R_{0}}(0)}$. Shifting $B_{r}(0)$ along $e_{1}$ on a sufficiently small distance $s\geqslant 0$, we get a contradiction as above. Indeed, Proposition 1.5, together with the invariance of ($\mathcal{D}$) upon orthogonal transformations of coordinates, implies that

[TABLE]

and corresponding minimizers generate a function from $\mathcal{M}$ which energy is strictly less than $\nu$. A contradiction.

Acknowledgments. The first author was supported by the project LO1506 of the Czech Ministry of Education, Youth and Sports. The second author was supported by the grant 17-01-00678 of Russian Foundation for Basic Research. The second author wishes to thank the University of West Bohemia, where this research was started, for the invitation and hospitality. The authors would like to thank A.I. Nazarov for stimulating discussions and valuable advices.

Appendix A Appendix

For readers’ convenience, in this section we sketchily show that the minimization problems $\mu_{\pm}(\Omega)$ and $\nu$ given by (1.2) and (1.8) possess minimizers which are least energy constant-sign and nodal solutions of ($\mathcal{D}$), respectively. (Note that the existence of “abstract” constant-sign and nodal solutions for problems of the type ($\mathcal{D}$) is known under much weaker assumptions on $f$, see, for instance, [17, 31]. However, we are interested in solutions with the least energy property.) Throughout this section, we always assume that $(A_{1})-(A_{4})$ are fulfilled.

First we need the following result about the geometry of the functional $E$.

Lemma A.1**.**

Let $u\in\accentset{\circ}{W}_{p}^{1}(\Omega)\setminus\{0\}$. Then there exists a unique constant $\alpha(u)\in(0,+\infty)$ such that $\alpha(u)u\in\mathcal{N}(\Omega)$. Moreover, $E[\alpha(u)u]=\max\limits_{\alpha>0}E[\alpha u]>0$.

Proof.

Define $Q:(0,+\infty)\to\mathbb{R}$ by $Q(\alpha)=E[\alpha u]$. Differentiating $Q$, we get

[TABLE]

Assume that there exists a critical point $\alpha_{1}>0$ of $Q$, i.e., $Q^{\prime}(\alpha_{1})=0$. Evidently, $\alpha_{1}u\in\mathcal{N}(\Omega)$. From the first part of $(A_{3})$ we deduce that $\frac{f(s)}{|s|^{p-2}s}$ is strictly decreasing for $s<0$ and strictly increasing for $s>0$. This implies that $Q^{\prime}(\alpha)>0$ for all $\alpha\in(0,\alpha_{1})$, and $Q^{\prime}(\alpha)<0$ for all $\alpha>\alpha_{1}$. Thus, any possible critical point of $Q$ on $(0,+\infty)$ is a point of a strict local maximum, and hence $Q$ has at most one critical point on $(0,+\infty)$. Moreover, since $Q(0)=0$, we get $Q(\alpha_{1})>0$.

Let us show that a critical point exists. In view of $(A_{1})-(A_{3})$, we apply [17, Theorem 17] to deduce that $Q(\alpha)>0$ for some $\alpha>0$ small enough. On the other hand, due to $(A_{1})$, $(A_{2})$, and $(A_{4})$, [17, Proposition 7] implies that $Q(\alpha)<0$ for $\alpha>0$ large enough. Therefore, since $Q(0)=0$, there exists a positive zero of $Q^{\prime}$. ∎

Arguing as in the first part of the proof of Lemma A.1, we deduce the following fact.

Lemma A.2**.**

Let $u\in\accentset{\circ}{W}_{p}^{1}(\Omega)$ and let $\Omega_{1}$ be a subdomain of $\Omega$. If $u\not\equiv 0$ a.e. in $\Omega_{1}$ and

[TABLE]

then

[TABLE]

Lemma A.3**.**

There exist minimizers of $\mu_{\pm}(\Omega)$ and $\nu$.

Proof.

Note first that $\mathcal{N}(\Omega)$ and $\mathcal{M}$ are nonempty. Indeed, let us take any nontrivial $u_{1},u_{2}\in\accentset{\circ}{W}_{p}^{1}(\Omega)\setminus\{0\}$ such that $u_{1}\geqslant 0$, $u_{2}\leqslant 0$, and $u_{1},u_{2}$ have disjoint supports. Then, Lemma A.1 yields the existence of $\alpha_{1},\alpha_{2}>0$ such that $\alpha_{1}u_{1},\alpha_{2}u_{2}\in\mathcal{N}(\Omega)$. Hence, there exist minimizing sequences for $\mu_{+}(\Omega)$ and $\mu_{-}(\Omega)$. Moreover, $\alpha_{1}u_{1}+\alpha_{2}u_{2}\in\mathcal{M}$, which implies the existence of a minimizing sequence for $\nu$.

Let us prove that a minimizing sequence for $\nu$ converges. The cases of $\mu_{\pm}(\Omega)$ can be treated analogously. Let $\{u_{k}\}_{k\in\mathbb{N}}\subset\mathcal{M}$ be a minimizing sequence for $\nu$. (The following technical details are reminiscent of the proof of Proposition 2.9.) First we show that $\{u_{k}\}_{k\in\mathbb{N}}$ is bounded. Note that from $(A_{2})$, $(A_{4})$, and the first part of $(A_{3})$ we get

[TABLE]

where $C_{1}>0$ does not depend on $k$. Therefore, supposing that $\|\nabla u_{k}\|_{L^{p}(\Omega)}\to+\infty$ as $k\to+\infty$ and recalling that $u_{k}\in\mathcal{M}$ for each $k\in\mathbb{N}$, we obtain

[TABLE]

since $\theta>p$. However, this fact contradicts a minimization nature of $\{u_{k}\}_{k\in\mathbb{N}}$. Thus, there exists $u\in\accentset{\circ}{W}_{p}^{1}(\Omega)$ such that, up to a subsequence, $u_{k}\to u$ and $u_{k}^{\pm}\to u^{\pm}$ weakly in $\accentset{\circ}{W}_{p}^{1}(\Omega)$ and strongly in $L^{q}(\Omega)$, $q\in(p,p^{*})$ (see [11, Section 3] and a direct generalization of [11, Lemma 2.3] to the case $p>1$). At the same time, due to $(A_{2})$ and the second part of $(A_{3})$, we can find $\mu\in(0,\lambda_{p}(\Omega))$ and $C_{2}>0$ such that $|f(s)|\leqslant\mu|s|^{p-1}+C_{2}|s|^{q-1}$ for all $s\in\mathbb{R}$. Therefore, we get

[TABLE]

for some $C_{3}>0$. If we suppose that $\|\nabla u_{k}^{+}\|_{L^{p}(\Omega)}\to 0$ as $k\to+\infty$, then for sufficiently large $k$ we obtain a contradiction since $\mu<\lambda_{p}(\Omega)$ and $q>p$. Thus, there exists $C_{4}>0$ such that $\|\nabla u_{k}^{+}\|_{L^{p}(\Omega)}>C_{4}$ for all $k$ large enough. Hence, $\int_{\Omega}u_{k}^{+}f(u_{k}^{+})\,dx>C_{4}$, which yields $u^{+}\not\equiv 0$ in $\Omega$. The same facts hold true for $-u_{k}^{-}$.

Now we show that $u^{\pm}_{k}\to u^{\pm}$ strongly in $\accentset{\circ}{W}_{p}^{1}(\Omega)$. By the weak convergence, we have

[TABLE]

Suppose, for instance, that $u^{+}_{k}$ does not converge strongly in $\accentset{\circ}{W}_{p}^{1}(\Omega)$, i.e., the strict inequality in (A.1) holds. Then Lemma A.1 implies the existence of $\alpha(u^{+}),\alpha(u^{-})>0$ such that $\alpha(u^{+})u^{+}\in\mathcal{N}(\Omega)$ and $-\alpha(u^{-})u^{-}\in\mathcal{N}(\Omega)$, and hence $\alpha(u^{+})u^{+}-\alpha(u^{-})u^{-}\in\mathcal{M}$. Moreover, $\alpha(u^{\pm})$ are unique points of maximum of $E[\alpha u^{\pm}]$ with respect to $\alpha>0$. Since each $u_{k}\in\mathcal{M}$, we also deduce from Lemma A.1 that $\alpha=1$ is a unique point of maximum of both $E[\alpha u_{k}^{+}]$ and $E[\alpha u_{k}^{-}]$ with respect to $\alpha>0$. Therefore,

[TABLE]

a contradiction. Consequently, $u_{k}^{\pm}\to u^{\pm}$ strongly in $\accentset{\circ}{W}_{p}^{1}(\Omega)$, $u\in\mathcal{M}$, and $E[u]=\nu$. ∎

Remark A.4**.**

As a corollary of Lemmas A.3 and A.1 we have $\mu_{\pm}(\Omega)>0$ and $\nu>0$.

Generalizing directly the proof of [7, Proposition 3.1] (see also [7, Proposition 6.1]) to the case $p>1$, we obtain the following result.

Lemma A.5**.**

Any minimizers of $\mu_{\pm}(\Omega)$ and $\nu$ are critical points of $E$ on $\accentset{\circ}{W}_{p}^{1}(\Omega)$, that is, weak solutions of ($\mathcal{D}$) with corresponding sign properties.

Remark A.6**.**

Any weak solution of ($\mathcal{D}$) belongs to $C^{1,\beta}(\overline{\Omega})$ for some $\beta\in(0,1)$, see [21, Corollary 1.1] and [30].

Remark A.7**.**

From the first part of $(A_{3})$ it follows that $f(0)=0$, $f(s)>0$ for $s>0$, and $f(s)<0$ for $s<0$. Hence, applying the strong maximum principle [38, Theorem 5], we derive that any weak constant-sign solution of ($\mathcal{D}$) is either strictly positive or strictly negative in $\Omega$, and has a nonzero normal derivative on the boundary $\partial\Omega$.

Remark A.8**.**

Let $u\in C^{1,\beta}(\overline{\Omega})$ be a positive weak solution of ($\mathcal{D}$). If $|\nabla u|>\eta$ in $\Omega_{\delta}:=\{x\in\Omega:\,\mathrm{dist}(x,\partial\Omega)<\delta\}$ for some $\eta,\delta>0$, then $u\in C^{2}(\overline{\Omega_{\delta}})$. See, e.g., [18, Lemma 5.2] with the source function $\widetilde{f}(x):=f(u(x))-au(x)^{p-1}$, $a<0$.

Remark A.9**.**

All the results stated above in Appendix A remain valid for problem ($\mathcal{D}$) in perturbed domains $\Omega_{t}=\Phi_{t}(\Omega)$, where the deformation $\Phi_{t}$ is given by (1.3), and $|t|<\delta$ with sufficiently small $\delta>0$. Indeed, the only assumption on the nonlinearity $f$ which depends on a domain is the second part of $(A_{3})$. However, since $\lambda_{p}(\Omega_{t})$ is continuous at $t=0$ (see [19]), we can take $\delta>0$ smaller (if necessary) and find $\widetilde{C}>0$ such that $\limsup\limits_{s\to 0}\frac{f(s)}{|s|^{p-2}s}<\widetilde{C}<\lambda_{p}(\Omega_{t})$ for all $|t|<\delta$. That is, the second part of $(A_{3})$ is satisfied uniformly for all $|t|<\delta$.