Estimates of Dirichlet heat kernels for unimodal L\'evy processes with low intensity of small jumps

TL;DR

This paper derives two-sided estimates for the transition densities of unimodal Lévy processes with low jump intensity, extending understanding of heat kernels in cases where the Lévy density's scaling index matches the Euclidean dimension.

Contribution

It provides the first two-sided Dirichlet heat kernel estimates for Lévy processes with Lévy densities that have a weak lower scaling index equal to the Euclidean dimension.

Findings

Establishes two-sided heat kernel estimates for a broad class of Lévy processes.

Covers cases with Lévy densities that are regularly varying with index equal to the dimension.

Extends previous results to processes with low jump intensity and matching scaling index.

Abstract

In this paper, we study transition density functions for pure jump unimodal L\'evy processes killed upon leaving an open set $D$. Under some mild assumptions on the L\'evy density, we establish two-sided Dirichlet heat kernel estimates when the open set $D$ is $C^{1, 1}$. Our result covers the case that the L\'evy densities of unimodal L\'evy processes are regularly varying functions whose indices are equal to the Euclidean dimension. This is the first results on two-sided Dirichlet heat kernel estimates for L\'evy processes such that the weak lower scaling index of the L\'evy densities is not necessarily strictly bigger than the Euclidean dimension.

Equations

$$\frac{f(R)}{f(r)}\ge c{\left(\frac{R}{r}\right)}^{{\alpha }_1}\text{for all}{c}_1<r\le R(\text{resp.}0<r\le R\le c_1).$$

$$\frac{f(R)}{f(r)}\le c{\left(\frac{R}{r}\right)}^{{\alpha }_2}\text{for all}{c}_2<r\le R(\text{resp.}0<r\le R\le c_2).$$

$$f(x)\asymp \underset{y\in [c_0,x]}{\sup }f(y)\text{for all}x>c_0.$$

$$f(x)\asymp \underset{y\in [c_0,x]}{\inf }f(y)\text{for all}x>c_0.$$

$$B(Q,R_{0})\cap D=\big{\{}y=(\widetilde{y},y_{d})\in B(0,R_{0})\mbox{ in }CS_{Q}:y_{d}>\Gamma(\widetilde{y})\big{\}}.$$

$$\displaystyle{\mathbb{E}}\Big{[}\exp\big{(}i\langle\xi,Y_{t}\rangle\big{)}\Big{]}=\int_{{\mathbb{R}}^{d}}e^{i\langle\xi,x\rangle}p(t,dx)=e^{-t\psi(\xi)},\quad\xi\in{\mathbb{R}}^{d},$$

$$\psi (\xi )={\int }_{{\mathbb{R}}^d}(1-\cos ⟨\xi ,x⟩)\nu (dx),\xi \in {\mathbb{R}}^d,$$

$${\kappa }_1{r}^{-d}\ell (r^{-1})\le \nu (r)\le {\kappa }_2{r}^{-d}\ell (r^{-1})\text{for all}r>0.$$

$${\mathbb{E}}\big{[}\exp(-\lambda S_{t})\big{]}=\exp\big{(}-t\varphi(\lambda)\big{)},\qquad\lambda\geq 0.$$

$$\nu (2r)\ge c\nu (r)\text{for all}r\in (0,R].$$

$$\nu (2r)\ge c\nu (r)\text{for all}r\in [R,\infty ).$$

$$K(r)$$

$$h(r)$$

$$K(r)\asymp r^{-2}{\int }_{|y|\le r}|y{|}^2\nu (y)dy\text{and}L(r)\asymp {\int }_{|y|>r}\nu (y)dy,$$

$${\ell }^{\ast }(r):=\underset{u\in [1,r]}{\sup }\ell (u)\text{for}r\ge 1$$

$${\ell }^{-1}(t):=\inf \{r\ge 1:{\ell }^{\ast }(r)>t\}\text{for}t>0.$$

$$\displaystyle c_{3}^{-1}\left(1\wedge\frac{1}{tL(\delta_{D}(x))}\right)^{1/2}\left(1\wedge\frac{1}{tL(\delta_{D}(y))}\right)^{1/2}t\nu(|x-y|)\exp\big{(}-c_{1}th(|x-y|)\big{)}$$

$$\displaystyle\leq p_{D}(t,x,y)\leq c_{3}\left(1\wedge\frac{1}{tL(\delta_{D}(x))}\right)^{1/2}\left(1\wedge\frac{1}{tL(\delta_{D}(y))}\right)^{1/2}t\nu(|x-y|)\exp\big{(}-c_{2}th(|x-y|)\big{)},$$

$$\displaystyle c_{6}^{-1}\left(1\wedge\frac{1}{tL(\delta_{D}(x))}\right)^{1/2}\left(1\wedge\frac{1}{tL(\delta_{D}(y))}\right)^{1/2}t\nu(\theta_{\eta}(|x-y|,t))\exp\big{(}-c_{4}th(\theta_{\eta}(|x-y|,t))\big{)}$$

$$\le p_D(t,x,y)$$

$$\displaystyle\leq c_{6}\left(1\wedge\frac{1}{tL(\delta_{D}(x))}\right)^{1/2}\left(1\wedge\frac{1}{tL(\delta_{D}(y))}\right)^{1/2}t\nu(\theta_{a_{0}}(|x-y|,t))\exp\big{(}-c_{5}th(\theta_{a_{0}}(|x-y|,t))\big{)},$$

$$\displaystyle c_{3}^{-1}L(\delta_{D}(x))^{-1/2}L(\delta_{D}(y))^{-1/2}\Big{(}\nu(|x-y|)\exp\big{(}-c_{1}th(|x-y|)\big{)}+\exp\big{(}-\kappa_{2}C_{4}th(r_{1}/2)\big{)}\Big{)}$$

$$\le p_D(t,x,y)$$

$$\displaystyle\leq c_{3}L(\delta_{D}(x))^{-1/2}L(\delta_{D}(y))^{-1/2}\Big{(}\nu(|x-y|)\exp\big{(}-c_{2}th(|x-y|)\big{)}+\exp\big{(}-\frac{\kappa_{1}C_{5}}{2}th(r_{2})\big{)}\Big{)},$$

$$c_4^{-1}{e}^{-{\lambda }_{1}t}L({\delta }_D(x){)}^{-1/2}L({\delta }_D(y){)}^{-1/2}\le p_D(t,x,y)\le c_4{e}^{-{\lambda }_{1}t}L({\delta }_D(x){)}^{-1/2}L({\delta }_D(y){)}^{-1/2},$$

$$\frac{{\kappa }_1{C}_5}{2}h(r_2)\le {\lambda }_1\le {\kappa }_2{C}_{4}h(r_1/2).$$

$$G_D(x,y):={\int }_0^{\infty }{p}_D(t,x,y)dt\text{for}x,y\in D.$$

$$G_D(x,y)$$

$$\displaystyle G_{D}(x,y)\asymp\bigg{(}1\wedge\frac{\varphi(|x-y|^{-2})}{\sqrt{\varphi(\delta_{D}(x)^{-2})\varphi(\delta_{D}(y)^{-2})}}\bigg{)}\frac{\varphi^{\prime}(|x-y|^{-2})}{|x-y|^{d+2}\varphi(|x-y|^{-2})^{2}}.$$

$$K(r)=r^{-2}{\int }_0^{r}s\ell (s^{-1})ds\asymp r^{-2}{\int }_{|y|\le r}|y{|}^2\nu (y)dy,$$

$$L(r)={\int }_r^{\infty }{s}^{-1}\ell (s^{-1})ds\asymp {\int }_{|y|>r}\nu (y)dy,$$

$$h(r)=K(r)+L(r)\asymp r^{-2}{\int }_{{\mathbb{R}}^d}\left(r^2\wedge |y{|}^2\right)\nu (y)dy.$$

$$K(r)\ge c\nu (r)r^{-2}{\int }_0^r{s}^{d+1}ds=c{r}^d\nu (r)\text{for all}r>0.$$

$$K(r)\asymp r^d\nu (r)\asymp \ell (r^{-1})\text{for}0<r\le 1$$

Full text

The research of Soobin Cho is supported by the POSCO Science Fellowship of POSCO TJ Park Foundation.

The research of Jaehoon Kang is supported by the National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT) (No. NRF-2019R1A5A1028324).

The research of Panki Kim is supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIP) : NRF-2016K2A9A2A13003815.

Keywords: transition density; heat kernel estimates; Dirichlet heat kernel estimates; unimodal Lévy processes; geometric stable process.

AMS 2020 Mathematics Subject Classification: 60J35, 60J50, 60J76

1. Introduction and Main results

1.1. Introduction

The transition density of a Lévy process killed upon leaving an open set $D$ (called the Dirichlet heat kernel of the process in $D$) is the fundamental solution of the equation $\partial_{t}u={\cal L}u$ with zero exterior condition on $D^{c}$, where ${\cal L}$ is the infinitesimal generator of the Lévy process. When the sample paths of the Lévy process are discontinuous, such generator is a non-local operator. Hence, transition densities of killed Lévy processes with jumps play significant roles in the study of non-local operators with killings. However, except for a few special cases, it is impossible to find an explicit expression of the Dirichlet heat kernel. Thus, obtaining sharp two-sided estimates on the Dirichlet heat kernels for discontinuous Lévy processes is a fundamental problem both in probability theory and analysis.

The first result on this topic was done in [14]. In [14], the third named author, jointly with Chen and Song, established sharp two-sided small time estimates on the Dirichlet heat kernel of an isotropic $\alpha$-stable process ($0<\alpha<2$) killed upon leaving a $C^{1,1}$ open set $D$ in ${\mathbb{R}}^{d}$. They also obtained large time estimates on the Dirichlet heat kernel when $D$ is bounded. After [14], much has been developed on the Dirichlet heat kernel estimates for discontinuous Markov processes. In [20], the authors obtained global Dirichlet heat kernel estimates for an isotropic $\alpha$-stable process $(0<\alpha<2)$ in a half-space like or an exterior $C^{1,1}$ open set $D$ in ${\mathbb{R}}^{d}$. Then in [6], the authors succeeded in extending that result for a $\kappa$-fat open subset $D$ in ${\mathbb{R}}^{d}$, and suggested a factorization formula for the Dirichlet heat kernel. Very recently, in [21], the first and third named authors, jointly with Song and Vondraček, obtained a general factorization formula for the Dirichlet heat kernel in a metric measure space. We refer to [12, 15, 19] for the Dirichlet heat kernel estimates for isotropic Lévy processes with non-vanishing Gaussian component, [16, 17] for relativistic stable processes (see also [22, 27]), and mixed $\alpha$-stable processes in [8, 18]. Note that, every aforementioned result on the Dirichlet heat kernel estimates cover neither processes with high intensity of small jumps nor processes with low intensity of small jumps, namely, whose Lévy measure has the form $\nu(dx)=|x|^{-d-2}\ell(|x|^{-1})dx$ or $\nu(dx)=|x|^{-d}\ell(|x|^{-1})dx$ for a function $\ell$ slowly varying at infinity (in Karamata’s sense). The case of high intensity of small jumps was treated for subordinate Brownian motions in [30] by the third named author jointly with Mimica using the result in [36]. After that, the third named author and Bae extended that result to a subordinate Brownian motion with non-vanishing Gaussian component in [1].

The purpose of this paper is treating the other case, low intensity of small jumps, in the study of the Dirichlet heat kernel estimates. We consider the Dirichlet heat kernel estimates on isotropic unimodal Lévy processes without Gaussian components whose Lévy measure has a radially non-increasing density which is comparable to $|x|^{-d}\ell(|x|^{-1})$ for a function $\ell$ satisfying weak scaling conditions at infinity with possibly non-positive lower scaling index (see Definition 1.1 for the notion of the weak scaling condition). Typical examples of such processes are geometric stable processes and iterated geometric stable processes. (See, e.g., [4, Page 112] for the definitions of these processes.) We refer to [24, 28, 29] for the scale invariant version of Harnack inequality and the Green function estimates for these processes. To the authors best knowledge, our result (Theorems 1.5 and 1.6) is the first result on the Dirichlet heat kernel estimates for Lévy processes with low intensity of small jumps both in small time and large time. Our paper is motivated by the recent paper [25] where sharp two-sided estimates on the heat kernel in the whole space ${\mathbb{R}}^{d}$ for pure jump isotropic unimodal Lévy processes (without killing) having the Lévy measure in the form $\nu(dx)=|x|^{-d}\ell(|x|^{-1})dx$ for a bounded function $\ell$ slowly varying at infinity. Unlike [25], in this paper, we allow the function $\ell$ to be unbounded and not slowly varying at infinity. Hence, our result is even new for the whole space case.

In this paper, we first derive heat kernel estimates for small time and the whole space by using the results and methods from [25]. Our heat kernel estimates in ${\mathbb{R}}^{d}$ have two forms depending on whether $\ell$ is bounded or unbounded. If the lower scaling index of $\ell$ is positive, then our results can be written in the form of $c_{1}(p(t,0)\wedge t\nu(x))\leq p(t,x)\leq c_{2}(p(t,0)\wedge t\nu(x))$, which coincides with the main result in [7]. Hereinafter, $p(t,x)$ denotes the transition density function which is also called as the heat kernel.

Next, we study behaviours of the process near the boundary of $D$. To do this, we use the boundary Harnack principle and gradient estimates on harmonic functions for pure jump isotropic Lévy processes. These results were obtained in [23] and [35], respectively, under some mild assumptions. Under a set of conditions that give the boundary Harnack principle and gradient estimates (see the condition (B) below), we obtain two-sided estimates on the mean exit time from the intersection of an open ball and $D$, and survival probability in $D$.

Using heat kernel estimates in the whole space and boundary behaviours of the process, we establish small time two-sided Dirichlet heat kernel estimates for isotropic unimodal Lévy processes in $C^{1,1}$ open sets (see Theorem 1.5 below). Even with the heat kernel estimates and the precise boundary behaviour of the mean exit time and the survival probability on hand, it is highly non-trivial to obtain Dirichlet heat kernel estimates because the lower scaling index of $\ell$ can be [math] and the heat kernel can be unbounded.

For bounded $C^{1,1}$ open sets in ${\mathbb{R}}^{d}$, we also obtain large time estimates (see Theorem 1.6 below). Since the killed semigroup $\{P_{t}^{D},t>0\}$ may not be compact operators for all $t>0$ even for a bounded open set $D\subset{\mathbb{R}}^{d}$, our method is different from ones for obtaining large time Dirichlet heat kernel estimates of stable processes or mixed stable processes.

Then we obtain two-sided estimates on the Green function in a bounded $C^{1,1}$ open subset in ${\mathbb{R}}^{d}$ (see Theorem 1.7 below). This result partially extends the result in [29] where the Green function estimtates on subordinate Brownian motions, whose Lévy-Khintchine exponent possibly has the lower scaling index [math], are treated (see Remark 1.8 below).

The paper ends with an explicit example on the Dirichlet heat kernel estimates and the Green function estimates for some Lévy processes including geometric stable processes and iterated geometric stable processes.

Notations: We will use the symbol “$:=$” to denote a definition, which is read as “is defined to be.” For $a,b\in{\mathbb{R}}$, we denote $a\wedge b:=\min\{a,b\}$ and $a\vee b:=\max\{a,b\}$. For two functions $f,g$ and a constant $c>0$, the notation $f\asymp g$ for $x>c$ means that there are strictly positive constants $c_{1}$ and $c_{2}$ such that $c_{1}g(x)\leq f(x)\leq c_{2}g(x)$ for all $x>c$. We denote an open ball by $B(x,r):=\{y\in{\mathbb{R}}^{d}:|x-y|<r\}$ and the diagonal set by $\text{diag}:=\{(x,x):x\in{\mathbb{R}}^{d}\}$. For an open set $D$ in ${\mathbb{R}}^{d}$, we denote $\delta_{D}(x):=\text{dist}(x,\partial D)$.

Upper case letters with subscripts $C_{i},i=0,1,2,\dots,$ and the constants $\kappa_{1},\kappa_{2},\alpha_{1}$ and $\alpha_{2}$ will remain the same throughout this paper. Lower case letters $c$’s without subscripts denote strictly positive constants whose values are unimportant and which may change even within a line, while values of lower case letters with subscripts $c_{i},i=0,1,2,\dots$, are fixed in each proof, and the labeling of these constants starts anew in each proof. The notation $c=c(a,b,\ldots)$ denotes a constant depending on $a,b,\ldots$.

1.2. Setup

To describe our results, we introduce the notions of the weak scaling conditions, almost monotonicity and some geometric properties of subsets of ${\mathbb{R}}^{d}$.

Definition 1.1**.**

Let $f:(0,\infty)\to(0,\infty)$ be a given (Lebesgue) measurable function.

(i) For $\alpha_{1}\in{\mathbb{R}}$ and $c_{1}>0$, we say that $f$ satisfies $\text{WLS}^{\infty}(\alpha_{1},c_{1})$ (resp. $\text{WLS}^{0}(\alpha_{1},c_{1})$) if there exists $c>0$ such that

[TABLE]

Similarly, for $\alpha_{2}\in{\mathbb{R}}$ and $c_{2}>0$, we say that $f$ satisfies $\text{WUS}^{\infty}(\alpha_{2},c_{2})$ (resp. $\text{WUS}^{0}(\alpha_{2},c_{2})$) if there exists $c>0$ such that

[TABLE]

If $f$ satisfies (1.1) (resp. (1.2)), then we call $\alpha_{1}$ (resp. $\alpha_{2}$) the lower scaling index (resp. the upper scaling index) of the function $f$. If $f$ satisfies both $\text{WLS}^{\infty}(\alpha_{1},c_{3})$ and $\text{WUS}^{\infty}(\alpha_{2},c_{3})$ for some $\alpha_{1},\alpha_{2}\in{\mathbb{R}}$ and $c_{3}>0$, we say that $f$ satisfies $\text{WS}^{\infty}(\alpha_{1},\alpha_{2},c_{3})$.

(ii) We say that $f$ is almost increasing if there exists $c_{0}>0$ such that

[TABLE]

Similarly, we say that $f$ is almost decreasing if there exists $c_{0}>0$ such that

[TABLE]

Definition 1.2**.**

(i) Let $d\geq 2$. An open set $D$ in ${\mathbb{R}}^{d}$ is said to be a (uniform) $C^{1,1}$ open set if there exist a localization radius $R_{0}>0$ and a constant $\Lambda>0$ such that for every $Q\in\partial D$, there exist a $C^{1,1}$ function $\Gamma=\Gamma_{Q}:{\mathbb{R}}^{d-1}\to{\mathbb{R}}$ satisfying $\Gamma(0)=0$, $\nabla\Gamma(0)=(0,\dots,0)$, $\|\nabla\Gamma\|_{\infty}\leq\Lambda$, $|\nabla\Gamma(\widetilde{x})-\nabla\Gamma(\widetilde{w})|\leq\Lambda|\widetilde{x}-\widetilde{w}|$ for $\widetilde{x},\widetilde{w}\in{\mathbb{R}}^{d-1}$ and an orthonormal coordinate system $CS_{Q}:y=(\widetilde{y},\,y_{d})$ with its origin at $Q$ such that

[TABLE]

The pair $(R_{0},\Lambda)$ is called the characteristics of the $C^{1,1}$ open set $D$.

(ii) An open set $D$ in ${\mathbb{R}}$ is said to be a $C^{1,1}$ open set if there exists a localization radius $R_{0}>0$ such that $D$ is an union of open intervals of length at least $R_{0}$ and distanced one from another at least $R_{0}$.

(iii) A bounded set $D$ in ${\mathbb{R}}^{d}$ is said to be of scale $(r_{1},r_{2})$ if there exist $x_{1},x_{2}\in{\mathbb{R}}^{d}$ such that $B(x_{1},r_{1})\subset D\subset B(x_{2},r_{2})$.

Let $Y=(Y_{t},t\geq 0)$ be a Lévy process in ${\mathbb{R}}^{d}$ with the Lévy-Khintchine exponent $\psi$. Then,

[TABLE]

where $p(t,dx)$ is the transition probability of $Y$. If $Y$ is a pure jump symmetric Lévy process with Lévy measure $\nu$, then $\psi$ is of the form

[TABLE]

where $\int_{{\mathbb{R}}^{d}}(1\wedge|x|^{2})\nu(dx)<\infty$.

A measure $\mu(dx)$ is isotropic unimodal if it is absolutely continuous on ${\mathbb{R}}^{d}\setminus\{0\}$ with a radial and radially non-increasing density. A Lévy process $Y$ is isotropic unimodal if $p(t,dx)$ is isotropic unimodal for all $t>0$. This is equivalent to the condition that the Lévy measure $\nu(dx)$ of $Y$ is isotropic unimodal if $Y$ is pure jump Lévy process. (See, [39].)

Throughout this paper, we always assume that $Y$ is a pure jump isotropic unimodal Lévy process with the Lévy-Khintchine exponent $\psi$. With a slight abuse of notation, we will use the notations $\psi(|x|)=\psi(x)$ and $\nu(dx)=\nu(x)dx=\nu(|x|)dx$ for $x\in{\mathbb{R}}^{d}$. Then, throughout this paper, we also assume the following condition (A) holds. A smooth function $\varphi:[0,\infty)\to[0,\infty)$ is called a Bernstein function if $(-1)^{n-1}\varphi^{(n)}(\lambda)\geq 0$ for all $n\geq 1$ and $\lambda>0$.

(A) The Lévy measure $\nu$ on ${\mathbb{R}}^{d}$ is infinite and there exist constants $\kappa_{1},\kappa_{2}>0$ and a continuous function $\ell:(0,\infty)\to(0,\infty)$ satisfying $\text{WS}^{\infty}(\alpha_{1},\alpha_{2},1)$ for some $-d<\alpha_{1}\leq\alpha_{2}<2$ such that

[TABLE]

If $d>1$, then we assume further that either $\alpha_{1}>-1$ or $\psi(\xi)=\varphi(|\xi|^{2})$ for a Bernstein function $\varphi$.

Note that, since we allow the constant $\alpha_{1}$ to be negative, the map $r\mapsto\ell(r^{-1})$ can be increasing near zero.

Here, we enumerate other main conditions which we will assume later.

(B) $\nu(r)$ is absolutely continuous such that $r\mapsto-\nu^{\prime}(r)/r$ is non-increasing on $(0,\infty)$ and there exists a constant $c_{0}>1$ such that $\nu(r)\leq c_{0}\nu(r+1)$ for all $r\geq 1$;

(C) $\ell(r)$ satisfies $\text{WUS}^{0}(\gamma,1)$ for some $\gamma<2$;

(S-1) $\limsup_{r\to\infty}\ell(r)<\infty$;

(S-2) $\limsup_{r\to\infty}\ell(r)=\infty$ and $\ell(r)$ is almost increasing;

(L-1) $\liminf_{r\to\infty}\ell(r)=0$ and $\ell(r)$ is almost decreasing;

(L-2) $0<\liminf_{r\to\infty}\ell(r)\leq\limsup_{r\to\infty}\ell(r)<\infty$;

(D) If $d=1$, then $\alpha_{2}<1$ where $\alpha_{2}$ is the constant in (A).

Remark 1.3**.**

Let $B=(B_{t},\,t\geq 0)$ be a Brownian motion in ${\mathbb{R}}^{d}$ and $S=(S_{t},\,t\geq 0)$ be a driftless subordinator independent of $B$. The process $X=(X_{t}:\,t\geq 0)$ defined by $X_{t}=B_{S_{t}}$ is called a subordinate Brownian motion (SBM). Every SBM is an isotropic unimodal Lévy process. Let $\varphi$ be the Laplace exponent of the subordinator $S$, namely,

[TABLE]

It is known that the Laplace exponent $\varphi$ is a Bernstein function with $\varphi(0)=0$. Since $S$ has no drift, $\varphi$ has the representation $\varphi(\lambda)=\int_{0}^{\infty}(1-e^{\lambda t})\mu(dt)$ where $\mu$ is a measure on $(0,\infty)$ satisfying $\int_{0}^{\infty}(1\land t)\mu(dt)<\infty$, called the Lévy measure of $\varphi$. Note that the characteristic exponent of $X$ is $\varphi(|\xi|^{2})$.

A function $f:(0,\infty)\to[0,\infty)$ is said to be completely monotone, if $(-1)^{n}f^{(n)}\geq 0$ on $(0,\infty)$ for every $n\geq 0$. A Bernstein function is said to be a complete Bernstein function, if its Lévy measure has a completely monotone density.

(i) Suppose that $\varphi$ is a complete Bernstein function such that $\lim_{\lambda\to\infty}\varphi(\lambda)=\infty$ and $\varphi^{\prime}$ satisfies $\text{WUS}^{\infty}(-\delta,1)$ for some $\delta\in(\frac{1}{2},1]$. Suppose further that, if $d=2$, then $\varphi^{\prime}$ satisfies $\text{WLS}^{\infty}(-\delta_{0},1)$ for some $\delta_{0}\in(0,2)$, and if $d=1$, then $\varphi^{\prime}$ satisfies $\text{WLS}^{\infty}(-\delta_{0},1)$ for some $\delta_{0}\in(\frac{1}{2},2\delta-\frac{1}{2})$. Then according to [29, Proposition 2.6], a SBM with the characteristic exponent $\varphi(|\xi|^{2})$ satisfies (A) with $\ell$ such that $\ell(r)\asymp r^{2}\varphi^{\prime}(r^{2})$ for $r\geq 1$.

(ii) Let $X$ be a SBM with the characteristic exponent $\varphi(|\xi|^{2})$ for a complete Bernstein function $\varphi$. Then by [22, Remark 1.4] and [5, Lemma 7.4], it satisfies (B). (See also the proof of [31, Proposition 3.5(b)].) **

Remark 1.4**.**

(i) (A) implies that $\nu(r)$ satisfies $\text{WLS}^{0}(-d-\alpha_{2},1)$. Therefore, under (A), for every $R>0$, there exists $c>0$ such that

[TABLE]

On the other hand, (C) implies that $\nu(r)$ satisfies $\text{WLS}^{\infty}(-d-\gamma,1)$ for some $\gamma<2$. Thus, (C) implies that for every $R>0$, there exists $c>0$ such that

[TABLE]

(ii) If (A) holds with $\alpha_{1}>0$, then (S-2) holds. (See, [3, Section 1.5].)

1.3. Main results.

We define for $r>0$,

[TABLE]

Since (A) holds, we see that

[TABLE]

which are the functions introduced in [37]. We also define

[TABLE]

and denote by $\ell^{-1}$ the right continuous inverse of $\ell^{*}$, that is,

[TABLE]

Now, we are ready to state our main results. Recall that $\delta_{D}(x)=\text{dist}(x,\partial D)$.

Theorem 1.5**.**

Suppose that $Y$ is a pure jump isotropic unimodal Lévy process satisfying (A) and (B). Let $D$ be a $C^{1,1}$ open set in ${\mathbb{R}}^{d}$ with characteristics $(R_{0},\Lambda)$. If $D$ is unbounded, we further assume that (C) holds. Then, the following estimates hold:

(i) If (S-1) holds, then for every $T>0$, there exist positive constants $c_{1}=c_{1}(d,\psi,T,R_{0},\Lambda)$, $c_{2}=c_{2}(d,\psi,T)$ and $c_{3}=c_{3}(d,\psi,T,R_{0},\Lambda)>1$ such that

[TABLE]

for all $(t,x,y)\in(0,T]\times(D\times D\setminus\text{diag})$.

(ii) If (S-2) holds, then for every $T>0$ and $\eta>0$, there exist positive constants $a_{0}=a_{0}(d,\psi)$, $c_{4}=c_{4}(d,\psi,T,R_{0},\Lambda),$ $c_{5}=c_{5}(d,\psi)$ and $c_{6}=c_{6}(d,\psi,T,\eta,R_{0},\Lambda)>1$ such that

[TABLE]

for all $(t,x,y)\in(0,T]\times D\times D$ where $\theta_{a}(r,t):=r\vee[\ell^{-1}(a/t)]^{-1}$ and $\ell^{-1}$ is defined as (1.6).

If we further assume that $D$ is bounded, then we can obtain the large time estimates for the Dirichlet heat kernel and the Green function estimates under some mild assumptions.

Theorem 1.6**.**

Suppose that $Y$ is a pure jump isotropic unimodal Lévy process satisfying (A) and (B). Let $D$ be a bounded $C^{1,1}$ open set in ${\mathbb{R}}^{d}$ with characteristics $(R_{0},\Lambda)$ of scale $(r_{1},r_{2})$. Then, the following estimates hold:

(i) If (L-1) holds, then for every $T>0$, there exist positive constants $c_{1}=c_{1}(d,\psi),$ $c_{2}=c_{2}(d,\psi)$ and $c_{3}=c_{3}(d,\psi,T,R_{0},\Lambda,r_{1},r_{2})>1$ such that

[TABLE]

for all $(t,x,y)\in[T,\infty)\times(D\times D\setminus\text{diag})$ where $\kappa_{1}$ and $\kappa_{2}$ are the positive constants in (A) and $C_{4}$ and $C_{5}$ are positive constants which only depend on the dimension $d$.

(ii) If (L-2) holds, then there exist $T_{1}\geq 0$ and $\lambda_{1}=\lambda_{1}(\psi,D)>0$ such that for every fixed $T>T_{1}$, there exists $c_{4}=c_{4}(d,\psi,T,R_{0},\Lambda,r_{1},r_{2})>1$ such that

[TABLE]

for all $(t,x,y)\in[T,\infty)\times D\times D$. Moreover, we have

[TABLE]

(iii) If (S-2) holds, then the estimates in (ii) holds with $T_{1}=0$. Moreover, the constant $-\lambda_{1}<0$ is the largest eigenvalue of the generator of $Y^{D}$.

For a Borel subset $D\subset{\mathbb{R}}^{d}$, the Green function $G_{D}(x,y)$ of $Y$ in $D$ is defined by

[TABLE]

Theorem 1.7**.**

Suppose that $Y$ is a pure jump isotropic unimodal Lévy process satisfying (A), (B) and (D). Let $D$ be a bounded $C^{1,1}$ open subset in ${\mathbb{R}}^{d}$ with characteristics $(R_{0},\Lambda)$ of scale $(r_{1},r_{2})$. Then, the Green function $G_{D}(x,y)$ of $Y$ in $D$ satisfies the following two-sided estimates: for every $x,y\in D$,

[TABLE]

where the comparison constants depend only on $d,\psi,R_{0},\Lambda$ and $r_{2}$.

Remark 1.8**.**

(i) One can obtain (1.7) just by integrating the estimates for $p_{D}(t,x,y)$ given in Theorems 1.5 and 1.6 (e.g. [30, Theorem 7.3]). However, to use Theorems 1.5 and 1.6, we need conditions more than (A), (B) and (D). By adopting arguments from [29] instead of integrating the Dirichlet heat kernel, we obtained the Green function estimates in more general situations.

(ii) It is established in [29, Theorem 1.2] that for a large class of transient subordinate Brownian motions, the Green function $G_{D}(x,y)$ in a bounded $C^{1,1}$ open set $D$ enjoys the following sharp two-sided estimates:

[TABLE]

An important novelty of this result is that it was the first explicit Green function estimates even if the lower scaling index of the Lévy-Khintchine exponent can be [math]. Note that, in view of Remark 1.3 and [28, Lemma 4.1], when the lower scaling index of the Lévy-Khintchine exponent can be [math], assumptions (A-1)–(A-5) in [29, Theorem 1.2] imply the following:

(1) The Lévy-Khintchine exponent $\psi(\xi)=\varphi(|\xi|^{2})$ for a complete Bernstein function $\varphi$. Thus, (B) holds (see Remark 1.3(ii));

(2) (A) holds with $\ell$ such that $\ell(r)\asymp r^{2}\varphi^{\prime}(r^{2})$ for $r\geq 1$, and constants $\alpha_{1}\in(-d,1)\cap[-2,1)$ and $\alpha_{2}<1$. Thus, (D) holds.

Therefore, by Lemma 2.1 and (2.4), we see that Theorem 1.7 recovers (1.8). Here, we note that Theorem 1.7 does not assume the transience of the process unlike [29, Theorem 1.2].

2. Heat kernel estimates in ${\mathbb{R}}^{d}$

Recall that under (A), we have

[TABLE]

Clearly, $L(r)$ is decreasing. Moreover, we see that $h^{\prime}(r)=-2r^{-1}K(r)\leq 0$ for all $r>0$ and hence $h(r)$ is also decreasing. Since the underlying process $Y$ is isotropic unimodal, there are a number of general properties related to these functions. (See, [7], [9] and [23].)

First, since $\nu(r)$ is non-increasing, we have

[TABLE]

On the other hand, by Karamata’s Tauberian-type theorem, the opposite inequality $K(r)\leq cr^{d}\nu(r)$ holds for $0<r\leq 1$ if and only if $\ell(r)$ satisfies $\text{WUS}^{\infty}(\gamma,1)$ for some $\gamma<2$. Similarly, we have $K(r)\leq cr^{d}\nu(r)$ for $r\geq 1$ if and only if $\ell(r)$ satisfies $\text{WUS}^{0}(\gamma^{\prime},1)$ for some $\gamma^{\prime}<2$. (See, [23, Appendix A].) In particular, (A) implies that

[TABLE]

and (C) implies that

[TABLE]

Next, by [7, (6) and (7)], there exist positive constants $C_{0}$ and $C_{1}$ which only depend on the dimension $d$ and $\kappa_{1}$ and $\kappa_{2}$ in (1.3) such that for all $r>0$,

[TABLE]

Under (A), we can extend this relations by including $L(r)$ if $r$ is small.

Lemma 2.1**.**

There exists a constant $c_{1}>0$ such that

[TABLE]

**Proof. **From the definitions of $L$ and $h$, the first inequality is obvious. To prove the second inequality, it suffices to show that there exists $c>0$ such that $L(r)\geq cK(r)$ for $0<r\leq 1$. Since (A) holds, by (1.4) and (2.2), we have $\nu(r)\asymp\nu(2r)$ and $K(r)\asymp r^{d}\nu(r)$ for $0<r\leq 1$. Thus, for $0<r\leq 1$, we get

[TABLE]

$\Box$

By Lemma 2.1 and (2.4), we deduce that $L(r)\asymp\psi(r^{-1})$ for small $r$. In view of this relation, to make some computations easier, we define $\Phi:[0,\infty)\to[0,\infty)$ by

[TABLE]

We used the change of variables $u=s^{-1}$ in the last equality.

Lemma 2.2**.**

*(i) $\Phi(r)$ satisfies $\text{WS}^{\infty}(\alpha_{1},\alpha_{2}\vee\frac{1}{2},1)$.

(ii) We have that*

[TABLE]

Moreover, there exists a constant $C_{2}>0$ such that

[TABLE]

**Proof. **(i) Let $\alpha_{2}^{\prime}=\alpha_{2}\vee\frac{1}{2}$. By the change of variables and (A), we have that

[TABLE]

The first inequality in (2.7) shows that $\Phi$ satisfies $\text{WLS}^{\infty}(\alpha_{1},1)$.

Now, we prove that $\Phi$ satisfies $\text{WUS}^{\infty}(\alpha_{2}^{\prime},1)$. Choose any $\kappa>1$ and $r\geq 2$. Let $n$ be the smallest integer satisfying $r^{n}\geq\kappa$. By applying the latter inequality in (2.7) $n$ times, since $\Phi(r)$ is increasing, we obtain

[TABLE]

Besides, for any $\kappa>1$ and $1<r<2$, since $\Phi$ is increasing, we see from the above inequalities that $\Phi(\kappa r)/\Phi(r)\leq(\Phi(2)/\Phi(1))\cdot(\Phi(2\kappa)/\Phi(2))\leq c_{3}\kappa^{\alpha_{2}^{\prime}}$. Hence, we get the desired result.

(ii) It follows from the definition of $\Phi$, Lemma 2.1 and (2.4). $\Box$

Let $C_{\infty}({\mathbb{R}}^{d})$ be the set of all continuous functions which vanish at infinity. In [26], Hartman and Wintner proved sufficient conditions in terms of the Lévy exponent $\psi$ under which the transition density $p(t,\cdot)$ of $Y$ is in $C_{\infty}({\mathbb{R}}^{d})$. Then, in [34], Knopova and Schilling improve that result and they also give some necessary conditions. Using (2.5) and (2.6), we can formulate these conditions in terms of $\Phi$. Since the underlying process $Y$ is isotropic unimodal, these conditions determine whether $p(t,0)<\infty$ or $p(t,0)=\infty$.

Proposition 2.3**.**

Let $p(t,\cdot)$ be the transition density of $Y$. Suppose that

[TABLE]

*Then, the following are true.

(i) If $c_{1}=\infty$, then $p(t,0)<\infty$ for all $t>0$.

(ii) If $c_{2}=0$, then $p(t,0)=\infty$ for all $t>0$.

(iii) If $0<c_{1}\leq c_{2}<\infty$, then there exist $T_{2}\geq T_{1}>0$ such that $p(t,0)=\infty$ for $0<t\leq T_{1}$ and $p(t,0)<\infty$ for $t>T_{2}$.*

*In particular, by l’Hospital’s rule, the following are true.

(iv) If $\;\liminf_{r\to\infty}\ell(r)=\infty$, then $p(t,0)<\infty$ for all $t>0$.

(v) If $\;\limsup_{r\to\infty}\ell(r)=0$, then $p(t,0)=\infty$ for all $t>0$.

(vi) If $0<\liminf_{r\to\infty}\ell(r)\leq\limsup_{r\to\infty}\ell(r)<\infty$, then there exist $T_{2}\geq T_{1}>0$ such that $p(t,0)=\infty$ for $0<t\leq T_{1}$ and $p(t,0)<\infty$ for $t>T_{2}$.*

**Proof. **By (2.5) and (2.6), the first two assertions follow from Part II in [26] and the third one follows from [34, Lemma 2.6]. $\Box$

Here, we introduce some general estimates which are established in [25]. Note that the following estimates hold no matter $p(t,0)<\infty$ or $p(t,0)=\infty$.

Proposition 2.4** ([25, Proposition 5.3]).**

There are constants $b_{0},c_{0}>0$, which only depend on the dimension $d$ and $\kappa_{2}$ in (1.3) such that for all $(t,x)\in(0,\infty)\times{\mathbb{R}}^{d}$,

[TABLE]

Proposition 2.5** ([25, Theorem 5.4]).**

There is a constant $c_{1}>0$, which only depends on the dimension $d$ and $\kappa_{2}$ in (1.3) such that for all $t>0$ and $x\in{\mathbb{R}}^{d}\setminus\{0\}$,

[TABLE]

The following lemma will be used several times to obtain heat kernel upper bounds for the whole space. (Cf. [25, Lemma 4.1 and Corollary 4.4].)

Lemma 2.6**.**

For every $\lambda>1$, there exists a constant $c=c(\lambda)>0$ such that

[TABLE]

**Proof. **Recall the condition (A). We first assume that either $d=1$ or $\alpha_{1}>-1$. For $y>0$, set $\nu_{1}(y)=\nu(y)$ if $d=1$, and

[TABLE]

We claim that there exists a constant $c_{1}>0$ such that

[TABLE]

If $d=1$, then (2.9) follows from (1.3). Hence, we assume $\alpha_{1}>-1$ and $d\geq 2$. Since $\ell$ is continuous and satisfies $\text{WS}^{\infty}(\alpha_{1},\alpha_{2},1)$, it also satisfies $\text{WS}^{\infty}(\alpha_{1},\alpha_{2},1/2)$. Hence, according to (1.3) and the change of the variables, we have that, for any $y\in(0,1]$,

[TABLE]

Besides, since $\nu$ is non-increasing and a Lévy measure, we also have that for any $y\in(0,1]$,

[TABLE]

Therefore, we obtain (2.9) with $c_{1}=c_{2}(2+\alpha_{1})/(1+\alpha_{1})+c_{5}$.

Observe that for $r>0$,

[TABLE]

Hence, we see that for any $1<k\leq\lambda$ and $r\geq 1$,

[TABLE]

By Taylor expansion of the cosine function, (2.9) and the assumption that $\ell$ satisfies $\text{WUS}^{\infty}(\alpha_{2},1)$ with $\alpha_{2}<2$, we have

[TABLE]

Next, to bound $I_{2}$ and $I_{3}$, we use a trick from the proof of [25, Theorem 3.5]. Since $y\mapsto\nu_{1}(y)$ is non-increasing, there exists a measure $-d\nu_{1}$ on $(0,\infty)$ such that $\nu_{1}(y)=\int_{y}^{\infty}(-d\nu_{1}(z))$ for $y>0$. Then by Fubini theorem and (2.9), we obtain

[TABLE]

Similarly, we also have that $I_{3}\leq 4c_{1}\ell(r)$. Therefore, we get (2.8) in this case.

For the case $\psi(\xi)=\varphi(|\xi|^{2})$ for a Bernstein function $\varphi$, we use [25, Lemma 5.13] and (2.2), and obtain that for any $1<k\leq\lambda$ and $r\geq 1$,

[TABLE]

This completes the proof. $\Box$

Now, we first consider the case when (S-2) holds. Recall that ${\ell^{*}}(r):=\sup_{u\in[1,r]}\ell(u)$ and $\ell^{-1}$ is the right continuous inverse of ${\ell^{*}}$ (see (1.6)). Since (S-2) holds, we get that $\lim_{r\to\infty}{\ell^{*}}(r)=\infty$ and there exists a constant $C_{3}\geq 1$ such that

[TABLE]

Note that in this case, by Proposition 2.3, $p(t,0)<\infty$ for all $t>0$. Here, we give the small time estimates for $p(t,0)$ under (S-2).

Lemma 2.7**.**

Assume that (S-2) holds. Then, there exists a constant $c_{1}>0$ such that

[TABLE]

for all $0<t\leq t_{1}$ and $x\in{\mathbb{R}}^{d}$ where $a_{1}:=2dC_{3}/C_{0}$, $b_{1}:=C_{0}/(4C_{2}C_{3})$ and $t_{1}:=a_{1}/{\ell^{*}}(3)$.

**Proof. **Let $a_{1}:=2dC_{3}/C_{0}$ and $t_{1}:=a_{1}/{\ell^{*}}(3)$. Then, $\ell^{-1}(a_{1}/t)\geq 3$ for all $t\in(0,t_{1}]$. By Fourier inversion theorem, (2.5), integration by parts and the change of variables $s=\Phi(r)$, we have that for all $t\in(0,t_{1}]$,

[TABLE]

Observe that for $\Phi(2)<v\leq u$, we have

[TABLE]

Thus, for all $\Phi(2)<v\leq u$, we have that (cf.   Section 3.10 in [3])

[TABLE]

Then, by (2.11) and the definition of $a_{1}$, we get

[TABLE]

On the other hand, define g(r):=r^{d}\exp\big{(}-\frac{C_{0}}{2C_{3}}t\Phi(r)\big{)} for $r\geq 1$. Then, we have

[TABLE]

It follows that $g$ is strictly increasing on $[1,\ell^{-1}(a_{1}/t))$. Therefore, we obtain

[TABLE]

We also have that

[TABLE]

Finally, we deduce the result from (2.6). $\Box$

Lemma 2.8**.**

Assume that (S-2) holds. Let $a_{1},b_{1}$ and $t_{1}$ be the positive constants in Lemma 2.7. Then, there exists a constant $c_{1}>0$ such that

[TABLE]

for all $0<t\leq t_{1}$ and $x\in{\mathbb{R}}^{d}$ satisfying $[\ell^{-1}(a_{1}/t)]^{-1}\leq|x|\leq 1/2$.

**Proof. **Fix $x\in{\mathbb{R}}^{d}$ satisfying $[\ell^{-1}(a_{1}/t)]^{-1}\leq|x|\leq 1/2$ and let $r=|x|$. By [25, (5.4)], the mean value theorem, (2.5) and Lemma 2.6, for $0<t\leq t_{1}$, we have

[TABLE]

Since $\ell$ satisfies $\text{WUS}^{\infty}(\alpha_{2},1)$ and $\Phi$ is increasing, we have

[TABLE]

On the other hand, define m(u):=u^{d-1/2}\exp\big{(}-\frac{C_{0}}{4C_{3}}t\Phi(u/r)\big{)} for $r>0$. Then, for all $u\in(r,1)$, since $1/r\leq\ell^{-1}(a_{1}/t)$ and $a_{1}=2dC_{3}/C_{0}$, we get

[TABLE]

It follows that $m(u)$ is increasing on $[r,1]$. In particular, we have that

[TABLE]

Since $C_{3}\geq 1$, we obtain

[TABLE]

Therefore, we deduce the result from (2), (2.13) and (2.6). $\Box$

In view of Lemma 2.7 and Lemma 2.8, we define for $a,r,t>0$,

[TABLE]

Note that both $r\mapsto\theta_{a}(r,t)$ and $t\mapsto\theta_{a}(r,t)$ are increasing, while $a\mapsto\theta_{a}(r,t)$ is decreasing.

Proposition 2.9**.**

Assume that (S-2) holds. For all $T>0$, there exists a constant $c_{1}>0$ such that for all $(t,x)\in(0,T]\times{\mathbb{R}}^{d}$,

[TABLE]

where $a_{1}$ and $b_{1}$ are the constants in Lemma 2.7.

**Proof. **Choose any $x\in{\mathbb{R}}^{d}$, and let $a_{1}$ and $t_{1}=a_{1}/\ell^{*}(3)$ be the constants in Lemma 2.7.

We first assume that $t\leq t_{1}$. If $|x|<[\ell^{-1}(a_{1}/t)]^{-1}$, then we have $\theta_{a_{1}}(|x|,t)=\ell^{-1}(a_{1}/t)^{-1}$ so that $\ell(\theta_{a_{1}}(|x|,t)^{-1})\geq a_{1}C_{3}^{-1}t^{-1}$ by (2.10). Hence, we obtain (2.15) from (2.1), (1.3) and Lemma 2.7. Else if $[\ell^{-1}(a_{1}/t)]^{-1}\leq|x|\leq 1/2$, then (2.15) follows from (2.1), (1.3), (2.10) and Lemma 2.8. Otherwise, if $|x|>1/2$, then since $r\mapsto h(r)$ is decreasing, we see that

[TABLE]

Thus, we get (2.15) from Proposition 2.5.

Now, suppose that $t\in(t_{1},T]$. In this case, we have that $\ell^{-1}(a_{1}/t)\asymp 1$. Therefore, if $|x|\geq[\ell^{-1}(a_{1}/t)]^{-1}$, then we get the result from Proposition 2.5. Otherwise, if $|x|<[\ell^{-1}(a_{1}/t)]^{-1}$, then by the semigroup property, Lemma 2.7 and (2.2), we have that

[TABLE]

This completes the proof. $\Box$

By combining Propositions 2.4 and 2.9, we obtain the following two-sided heat kernel estimates under (S-2).

Corollary 2.10**.**

Assume that (S-2) holds. For all $T>0$, there exists a constant $c_{1}>1$ such that for every fixed $\delta>0$, we have that for all $(t,x)\in(0,T]\times{\mathbb{R}}^{d}$,

[TABLE]

where $b_{0}$ is the constant in Proposition 2.4, and $a_{1}$ and $b_{1}$ are the constants in Lemma 2.7.

**Proof. **The upper bound follows from Proposition 2.9. On the other hand, since $p(t,\cdot)$ is radially non-increasing and $\theta_{\delta}(|x|,t)\geq|x|$ for all $\delta,t>0$ and $x\in{\mathbb{R}}^{d}$, we deduce the lower bound from Proposition 2.4. $\Box$

Remark 2.11**.**

If $\ell$ satisfies $\text{WLS}^{\infty}(\alpha,1)$ for some $\alpha>0$, then $\ell(r)\asymp\Phi(r)$ for $r\geq 1$. (See, [3, Theorem 2.6.1].) Therefore, when $\ell$ satisfies $\text{WLS}^{\infty}(\alpha,1)$ for some $\alpha>0$, we see that the estimate (2.10) can be expressed as follows: For every $(t,x)\in(0,T]\times{\mathbb{R}}^{d}$,

[TABLE]

Hence, if (C) further holds, then we see from (2.2) and (2.3) that $p(t,x)\asymp\Phi^{-1}(1/t)^{d}\wedge t\nu(|x|)$ for $(t,x)\in(0,T]\times{\mathbb{R}}^{d}$. In view of (2.4), (2.5) and (2.6), this coincides with the main result in [7]. **

In the rest of this section, we assume that (S-1) holds. Then, by Proposition 2.3, we have that $p(t,0)=\infty$ for all sufficiently small $t$. Recently, some general estimates for such type of heat kernels were established in [25]. Using that results, we obtain the heat kernel estimates in analogous form to (2.10).

Proposition 2.12**.**

Assume that (S-1) holds. Then, there exist constants $t_{0},c_{1}>0$ such that for all $(t,x)\in(0,t_{0}]\times({\mathbb{R}}^{d}\setminus\{0\})$,

[TABLE]

**Proof. **Let $\omega(r)=K(1){\bf 1}_{\{0<r\leq 1\}}(r)+K(r^{-1}){\bf 1}_{\{r>1\}}(r)$ for $r>0$ where ${\bf 1}_{A}$ denotes the indicator function on a set $A$. By (2.2), Lemma 2.6, (A) and (S-1), there exists a constant $c_{0}>0$ such that $c_{0}\omega(r)$ satisfies the assumptions (5.7) and (5.8) in [25]. Therefore, by [25, Proposition 5.6], there exist $t_{0},c_{1}>0$ such that for all $t\in(0,t_{0}]$ and $0<|x|<1$, the estimate (2.17) holds. Moreover, for $t\in(0,t_{0}]$ and $|x|\geq 1$, we have that $e^{-t\psi(|x|^{-1})}\asymp 1$. Then, we get the result from Proposition 2.5. $\Box$

Corollary 2.13**.**

Assume that (S-1) holds. For all $T>0$, there exist constants $c_{1},b_{2}>0$ such that

[TABLE]

for all $(t,x)\in(0,T]\times({\mathbb{R}}^{d}\setminus\{0\})$ where $b_{0}$ is the constant in Proposition 2.4.

**Proof. **By Propositions 2.4 and 2.12, (2.4) and induction, it suffices to prove the upper bound in (2.18) for $t\in(t_{0},2t_{0}]$ and $x\in{\mathbb{R}}^{d}\setminus\{0\}$, where $t_{0}$ is the constant in Proposition 2.12. If $|x|\geq 1$, then \exp\big{(}-cth(|x|)\big{)}\asymp 1 for each fixed constant $c>0$ so that the assertion holds by Proposition 2.5. Suppose that $|x|<1$. Without loss of generality, we may assume that $2b_{2}\leq b_{0}$. Then, by the semigroup property, (2.2), the induction hypothesis, monotonicity of $p(t,\cdot)$ and Proposition 2.5, we get

[TABLE]

$\Box$

3. Boundary Harnack principle with explicit decay

In this section, we investigate the boundary behaviour of the process via the renewal function $V$ of $Y$ and the tail of its Lévy measure. Throughout this section, we assume that (B) holds. For an open set $D\subset{\mathbb{R}}^{d}$, the first exit time is denoted by $\tau_{D}:=\inf\{t>0:\,Y_{t}\notin D\}$. We give the probabilistic definition of a (regular) harmonic function.

Definition 3.1**.**

(i) A function $u:{\mathbb{R}}^{d}\to{\mathbb{R}}$ is said to be harmonic in an open set $D\subset{\mathbb{R}}^{d}$ with respect to $Y$ if for every open set $B$ whose closure is a compact subset of $D$, ${\mathbb{E}}_{x}[|u(Y_{\tau_{B}})|]<\infty$ and $u(x)={\mathbb{E}}_{x}[u(Y_{\tau_{B}})]$ for every $x\in B$.

(ii) A function $u:{\mathbb{R}}^{d}\to{\mathbb{R}}$ is said to be regular harmonic in an open set $D\subset{\mathbb{R}}^{d}$ with respect to $Y$ if ${\mathbb{E}}_{x}[|u(Y_{\tau_{D}})|]<\infty$ and $u(x)={\mathbb{E}}_{x}[u(Y_{\tau_{D}})]$ for every $x\in D.$

Here, we provide the precise definition of the renewal function $V$ of $Y$. Let $Y^{d}$ be the last coordinate of $Y$, $M_{t}=\sup_{s\leq t}Y^{d}_{s}$ and ${\mathscr{L}}_{t}$ be the local time at [math] for $M_{t}-Y^{d}_{t}$, the last coordinate of $Y$ reflected at the supremum. Define the ascending ladder-height process as $H_{t}=Y_{{\mathscr{L}}^{-1}_{t}}^{d}=M_{{\mathscr{L}}^{-1}_{t}}$ where ${\mathscr{L}}^{-1}$ is the right continuous inverse of ${\mathscr{L}}$. Then, the renewal function $V$ is defined as

[TABLE]

Since the process $Y$ is isotropic unimodal, there are several known properties for the renewal function. (See, [38, Theorem 1.2], [2, p.74] and [8, Section 1.2].)

Lemma 3.2**.**

*(i) $V$ is strictly increasing, $V(s)=0$ if $s<0$ and $\lim_{s\to\infty}V(s)=\infty$.

(ii) $V$ is subadditive; that is,*

[TABLE]

(iii) $V$ is absolutely continuous and harmonic on $(0,\infty)$ for the process $Y_{t}^{d}$. Also, $V^{\prime}$ is a positive harmonic function for $Y_{t}^{d}$ on $(0,\infty)$.

According to [9, Proposition 2.4], the relation (2.4) can be extended to include the renewal function. That is, there exist comparison constants which are only depend on the dimension $d$ and $\kappa_{1}$ and $\kappa_{2}$ in (1.3) such that $h(r)\asymp\psi(r^{-1})\asymp[V(r)]^{-2}$ for all $r>0$. Then, by Lemmas 2.1 and 2.2, we have that

[TABLE]

In particular, by (3.1) and Lemma 2.2, there are constants $c_{1},c_{2},c_{3},c_{4}>0$ such that

[TABLE]

and

[TABLE]

Proposition 3.3**.**

The renewal function $V$ is twice-differentiable on $(0,\infty)$, and there exists $c_{1}>0$ such that

[TABLE]

**Proof. **Since (A) and (B) hold, the scale-invariant Harnack inequality holds for $Y$. (See, [23, Theorem 1.9].) Then, the results follows from [35, Theorem 1.1] and Lemma 3.2(iii). $\Box$

Define $w(x):=V((x_{d})^{+})$ for $x\in{\mathbb{R}}^{d}$ and let $\mathbb{H}:=\{x=(\widetilde{x},x_{d})\in{\mathbb{R}}^{d}:x_{d}>0\}$ the upper half-space. Since the renewal function $V$ is harmonic on $(0,\infty)$ for $Y^{d}$, by the strong Markov property, $w$ is harmonic in $\mathbb{H}$ with respect to $Y$.

Proposition 3.4**.**

For all $\lambda>0$, there exists $c_{1}=c_{1}(d,\lambda)>0$ such that for any $r>0$,

[TABLE]

**Proof. **See, the proof of [22, Proposition 3.2]. $\Box$

Denote $C_{\infty}^{2}({\mathbb{R}}^{d})$ by the set of all twice-differentiable functions in ${\mathbb{R}}^{d}$ vanishing at infinity. We define an operator ${\cal L}_{Y}$ as follows: for $\varepsilon>0$ and $x\in{\mathbb{R}}^{d}$,

[TABLE]

Theorem 3.5**.**

For any $x\in\mathbb{H}$, ${\cal L}_{Y}w(x)$ is well-defined and ${\cal L}_{Y}w(x)=0$.

**Proof. **By Propositions 3.3 and 3.4, using [11, Lemma 2.3, Theorem 2.11], the proof is essentially the same as the one given in [22, Theorem 3.3]. Hence, we omit it. $\Box$

Lemma 3.6**.**

Let $D$ be a $C^{1,1}$ open set in ${\mathbb{R}}^{d}$ with characteristics $(R_{0},\Lambda)$. For any $Q\in\partial D$ and $r>0$, we define

[TABLE]

Then, there exist $R_{1}=R_{1}(R_{0},\Lambda,\psi,d)\in(0,(R_{0}\wedge 1)/2]$ and $c_{1}=c_{1}(R_{0},\Lambda,\psi,d)>1$ independent of $Q$ such that for every $r\in(0,R_{1})$, ${\cal L}_{Y}h_{r}$ is well defined in $D\cap B(Q,r/4)$ and

[TABLE]

**Proof. **Since the case of $d=1$ is easier, we only give the proof for $d\geq 2$. Fix $Q\in\partial D,r\in(0,(R_{0}\wedge 1)/2)$ and $x\in D\cap B(Q,r/4)$. Let $z\in\partial D$ be the point satisfying $\delta_{D}(x)=|x-z|$ and denote $\Gamma_{z}$ and $CS_{z}$ by the $C^{1,1}$ function and orthonormal coordinate system determined by $z$, respectively. (See, Definition 1.2.) Henceforth, we use the coordinate system $CS_{z}$. Hence, we have $z=0$, $x=(\widetilde{0},x_{d})$ and $D\cap B(z,R_{0})=\{y=(\widetilde{y},y_{d})\in B(0,R_{0})\;\;\text{in}\;\;CS_{z}:y_{d}>\Gamma_{z}(\widetilde{y})\}.$ Since $D$ is a $C^{1,1}$ open set, it satisfies the inner and outer ball conditions. Thus, we may assume that

[TABLE]

and

[TABLE]

where $\phi:{\mathbb{R}}^{d-1}\to{\mathbb{R}}$ is defined by $\phi(\widetilde{y}):=1-\sqrt{1-|\widetilde{y}|^{2}}.$

Let $E:=\{y=(\widetilde{y},y_{d}):|\widetilde{y}|<r/2,|y_{d}|<r/2\}$, $E_{1}:=\{y\in E:y_{d}>2\phi(\widetilde{y})\}$ and $E_{2}:=\{y\in E:y_{d}<-2\phi(\widetilde{y})\}$. We also let $w_{z}(y):=V((y_{d})^{+})$. By Theorem 3.5, we get ${\cal L}_{Y}w_{z}(x)=0$. Since $h_{r}(x)=w_{z}(x)$ and $h_{r}(y)=w_{z}(y)=0$ for $y\in E_{2}$, we have

[TABLE]

First, since $|h_{r}(y)|\leq V(r)$, using Lemma 2.1, (3.1), (3.2) and Proposition 3.4, we have

[TABLE]

Next, we note that for $y\in E\setminus(E_{1}\cup E_{2})$,

[TABLE]

and hence by subadditivity of $V$, we obtain

[TABLE]

Since $1-\sqrt{1-l^{2}}\leq l^{2}$ for $0\leq l<1$, we have for $0<s<1$,

[TABLE]

where $m_{d-1}(dx)$ is the $(d-1)$-dimensional Lebesgue measure. From these observations, using (1.4), (2.1), the definitions of $K$ and $h$, (3.1) and (3.2), since $\alpha_{2}<2$, we get

[TABLE]

Lastly, to estimate $I_{1}$ we first claim that

[TABLE]

Indeed, for any $y\in E_{1}$, if $0<y_{d}\leq\delta_{D}(y)$, then we have

[TABLE]

Otherwise, if $\delta_{D}(y)<y_{d}$, then we have

[TABLE]

Hence, since $|\widetilde{y}|,|y_{d}|<r/2<1/4$, we get $y_{d}-\delta_{D}(y)\leq(2/3)|\widetilde{y}|^{2}\leq(4/3)\phi(\widetilde{y})<(2/3)y_{d}$. Therefore, (3.5) holds.

Recall that by Lemma 3.2(iii), $V^{\prime}$ is a harmonic function for $Y^{d}_{t}$ on $(0,\infty)$. Since the scale-invariant Harnack inequality holds for $Y$ (see, [23, Theorem 1.9]), by (3.5), we deduce that for every $y\in E_{1}$,

[TABLE]

for some constant $c>0$ independent of choice of $Q,r$ and $x$. Hence, we obtain

[TABLE]

By the monotonicity of $\nu$, (2.1), the definition of $h$, (3.1) and (3.2), since $\alpha_{2}<2$, we have

[TABLE]

Besides, set $\rho:=4^{-1}((2-\alpha_{2})\wedge 1)\in(0,1/4]$. By Proposition 3.3, (2.1), the definition of $h$, (3.1) and (3.2), since $\sqrt{a^{2}+b^{2}}\geq(a+b)/\sqrt{2}$ for all $a,b\geq 0$, we have that

[TABLE]

By (3.2), since $1-\rho>(\alpha_{2}/2)\vee(1/4)$, we see that for all $u\in[y_{d},1]$,

[TABLE]

Since $s\mapsto\sup_{u\in[s,1]}(V(u)u^{\rho-1})$ and $s\mapsto s^{-\rho}V(s)^{-2}$ are non-increasing, by (3.6), [33, Lemma 4.4], (3.7) and (3.2), we obtain that

[TABLE]

Finally, we get from (3) that $|{\cal L}_{Y}h_{r}(x)|\leq cV(r)^{-1}$. By (3.1), this finishes the proof. $\Box$

For $l>0$, we define $D_{int}(l):=\{y\in D:\delta_{D}(y)>l\}$.

Lemma 3.7**.**

Let $D$ be a $C^{1,1}$ open set in ${\mathbb{R}}^{d}$ with characteristics $(R_{0},\Lambda)$ and $R_{1}$ be the constant in Lemma 3.6. Then, there exist constants $R_{2}=R_{2}(R_{0},\Lambda,\psi,d)\in(0,R_{1}/16]$ and $c_{1}=c_{1}(R_{0},\Lambda,\psi,d)>1$ such that for every $r\in(0,R_{2}]$ and $x\in D$ with $\delta_{D}(x)<r/2$,

[TABLE]

and

[TABLE]

where $z\in\partial D$ is the point satisfying $\delta_{D}(x)=|x-z|$.

**Proof. **Let $R_{1}$ be the constant in Lemma 3.6. Fix $r\in(0,R_{2}]$ and $x\in D$ with $\delta_{D}(x)<r/2$ where the constant $R_{2}\in(0,R_{1}/16]$ will be selected later. Let $z\in\partial D$ be the point satisfying $\delta_{D}(x)=|x-z|$. As in Lemma 3.6, we denote by $\Gamma_{z}:{\mathbb{R}}^{d-1}\to{\mathbb{R}}$ and $CS_{z}$ for a $C^{1,1}$ function and coordinate system with respect to $z$, respectively and hereinafter we use the coordinate system $CS_{z}$.

Denote by $U(s):=D\cap B(0,s)$ for $s>0$. Then, we define

[TABLE]

Using Dynkin’s formula and approximation argument, (see, [30, Proposition 4.7],) by Lemma 3.6, there exists a positive constant $a$ independent of choice of $R_{2}$ and $x$ such that

[TABLE]

for every open subset $W\subset U(R_{1}/8)$.

Let

[TABLE]

We claim that ${\cal C}_{2}\subset{\cal C}_{1}\subset D$. Indeed, the first inclusion is obvious. Moreover, for all $y\in{\cal C}_{1}$, $y_{d}-\Gamma_{z}(\widetilde{y})\geq y_{d}-\Lambda|\widetilde{y}|\geq y_{d}/2>0.$ Hence, the second inclusion holds.

Observe that for $0<s\leq R_{1}$ and $y\in{\cal C}_{2}\cap\partial U(s)$, we have

[TABLE]

for some constant $c_{0}$ which only depends on $\Lambda$. By the Lévy system, (3.11), integration by parts, Lemma 2.1, (3.1), (3.3) and monotonicity of $V$, for $0<4s<R_{1}$,

[TABLE]

for some constants $c_{1},c_{2},c_{3}>0$ independent of $s$. Moreover, by the similar argument, we also have that

[TABLE]

and

[TABLE]

We used (1.4) in the third inequality.

For selected constants $a,c_{1},c_{2},c_{3},c_{4}$ and $c_{5}$ in (3.10), (3) and (3), we set

[TABLE]

Then, by (3.10) and (3), we get

[TABLE]

This proves the upper bound of (3.8).

On the other hand, by (3), we get

[TABLE]

By [9, Lemma 2.1] and (3.1), there exists $c_{6}>0$ such that

[TABLE]

Then, by (3.10), (3), (3.16), (3.2) and the monotonicity of $V$, we get

[TABLE]

This proves the lower bound of (3.8) in view of (3.1). Finally, we get (3.9) from (3.15). $\Box$

4. Estimates of survival probability

In this section, we obtain two-sided estimates for the survival probability ${\mathbb{P}}_{x}(\tau_{D}>t)$ which play a crucial role in factorization of the Dirichlet heat kernel. We first state the general two-sided estimates for the survival probability in balls which are recently established in [25].

Proposition 4.1** ([25, Proposition 5.2]).**

There exist positive constants $c_{1},c_{2},C_{4}$ and $C_{5}$ which only depend on the dimension $d$ such that for all $t,r>0$,

[TABLE]

where $\kappa_{1}$ and $\kappa_{2}$ are constants in (A). As a consequence, for all $r>0$,

[TABLE]

Note that the last inequality in (4.1), and (4.2) were obtained for a large class of Feller processes in ${\mathbb{R}}^{d}$. See [10, Corollaries 5.3, 5.8 and Theorem 5.9].

In the rest of this section, we assume that (B) holds. Fix $T>0$ and $D$ a $C^{1,1}$ open set in ${\mathbb{R}}^{d}$ with characteristics $(R_{0},\Lambda)$. Let $R_{2}$ be the constant in Lemma 3.7. For $t\in(0,T]$, we set

[TABLE]

For $x\in D$ with $\delta_{D}(x)<r_{t}/2$, we define an open neighborhood $U(x,t)$ of $x$ and an open ball $W(x,t)\subset D\setminus U(x,t)$ as follows:

*Find $z_{x}\in\partial D$ satisfying $\delta_{D}(x)=|x-z_{x}|$ and let $v_{x}:=z_{x}+2r_{t}(x-z_{x})/|x-z_{x}|$. Then, we have $\delta_{D}(v_{x})\geq r_{t}/\sqrt{1+\Lambda^{2}}.$ We define *

[TABLE]

Note that by the construction, we have that for all $u\in U(x,t)$ and $w\in W(x,t)$,

[TABLE]

and

[TABLE]

It follows that

[TABLE]

Proposition 4.2**.**

Let $D$ be a $C^{1,1}$ open set in ${\mathbb{R}}^{d}$ with characteristics $(R_{0},\Lambda)$. Let $r_{t}$ and $U(x,t)$ be defined as in just before the Proposition. For all $T>0$ and $M\geq 1$, we have that for every $t\in(0,T]$ and $x\in D$ with $\delta_{D}(x)<r_{t}/2$,

[TABLE]

where the comparison constants depend only on $T,M,\psi,R_{0},\Lambda$ and $d$.

**Proof. **Recall that $z_{x}\in\partial D$ is the point satisfying $\delta_{D}(x)=|x-z_{x}|$. Let

[TABLE]

Indeed, we have $r_{t}/(2\sqrt{1+\Lambda^{2}})\leq\delta_{D}(o_{x})\leq r_{t}/2.$ By (A) and (B), we see that assumptions in [23, Theorem 1.9] hold and hence by that theorem, the (scale-invariant) boundary Harnack principle holds. Therefore, we get

[TABLE]

where $W(x,t)$ is the subset of $D$ defined as in just before the Proposition. By the Lévy system, (1.4), (4.4) and Lemma 3.7, we have

[TABLE]

Similarly, we also have that ${\mathbb{P}}_{o_{x}}(Y_{\tau_{U(x,t)}}\in W(x,t))\asymp{\mathbb{E}}_{o_{x}}[\tau_{U(x,t)}]\nu(r_{t})r_{t}^{d}\asymp L(r_{t})^{-1}\nu(r_{t})r_{t}^{d}.$

Then, by the strong Markov property, Chebyshev’s inequality, (4.5) and Lemma 3.7, since $L(r_{t})\asymp t^{-1}$, we obtain

[TABLE]

On the other hand, for any $a>0$, by the strong Markov property, (4.1), (3.1), Lemma 3.7 and Chebyshev’s inequality,

[TABLE]

Take $a=(2c_{2}^{-1})\vee M$. By Lemma 3.7 and the fourth line in the above inequalities, we obtain

[TABLE]

This completes the proof. $\Box$

Corollary 4.3**.**

Let $D$ be a $C^{1,1}$ open set in ${\mathbb{R}}^{d}$ with characteristics $(R_{0},\Lambda)$. For all $T>0$, there exists a constant $c_{1}=c_{1}(d,T,\psi,R_{0},\Lambda)>1$ such that for every $t\in(0,T]$ and $x\in D$,

[TABLE]

**Proof. **We use the same notations as those in Proposition 4.2. If $\delta_{D}(x)<r_{t}/2$, then the result follows from Proposition 4.2. If $\delta_{D}(x)\geq r_{t}/2$, then $tL(\delta_{D}(x))\leq tL(r_{t}/2)\leq c$. Also, by (4.1) and (3.1), we get $1\geq{\mathbb{P}}_{x}(\tau_{D}>t)\geq{\mathbb{P}}_{x}(\tau_{B(x,r_{t}/2)}>t)\geq c$. $\Box$

Corollary 4.4**.**

Let $D$ be a bounded $C^{1,1}$ open subset in ${\mathbb{R}}^{d}$ with characteristics $(R_{0},\Lambda)$ of scale $(r_{1},r_{2})$. Then, there exists $c_{1}=c_{1}(R_{0},\Lambda,\psi,d)>0$ such that for all $t>0$ and $x\in D$,

[TABLE]

where $\kappa_{1},\kappa_{2}$ are constants in (A) and $C_{4},C_{5}$ are constants in (4.1).

**Proof. **Fix $(t,x)\in(0,\infty)\times D$. If $t\leq 2$, then the assertion follows from Corollary 4.3. Hence, we assume that $t>2$. Let $x_{1},x_{2}\in{\mathbb{R}}^{d}$ be the points satisfying $B(x_{1},r_{1})\subset D\subset B(x_{2},r_{2})$. By the semigroup property, (4.1) and Corollary 4.3, we get

[TABLE]

To prove the lower bound, we first assume that $\delta_{D}(x)<R_{2}/2$ where $R_{2}$ is the constant in Lemma 3.7. Without loss of generality, we may assume that $R_{2}\leq r_{1}/2$. Let $z\in\partial D$ be the point satisfying $\delta_{D}(x)=|x-z|$ and $\theta$ be the shift operator defined as $Y_{t}\circ\theta_{s}=Y_{s+t}$. Then, by the strong Markov property, (3.9), the Lévy system and (4.1), we have

[TABLE]

Indeed, on $\{Y_{\tau_{D\cap B(z,R_{2})}}\in D_{int}(R_{2}/4)\}$, we have $B(Y_{\tau_{D\cap B(z,R_{2})}},R_{2}/4)\subset D$. Also, since $R_{2}\leq r_{1}/2$, we can always find $A\subset B(x_{1},r_{1}/2)\setminus B(Y_{\tau_{D\cap B(z,R_{2})}},R_{2}/2)$ such that $|A|\geq c_{1}>0$ for some constant $c_{1}>0$. Then, by the Lévy system and (4.2), we obtain

[TABLE]

Similarly, if $\delta_{D}(x)\geq R_{2}/2$, then we have

[TABLE]

$\Box$

5. Small time Dirichlet heat kernel estimates in $C^{1,1}$ open set

In this section, we provide the proof of Theorem 1.5. Let $T>0$ be a fixed constant and $D$ be a fixed $C^{1,1}$ open set in ${\mathbb{R}}^{d}$ with characteristics $(R_{0},\Lambda)$. We assume that (B) holds. If $D$ is unbounded, then we further assume that (C) holds. Then, by (1.4) and (1.5), we have

[TABLE]

By (2.2), (2.3), Corollary 2.10 and Corollary 2.13, we have the following heat kernel estimates for small $t$. Let $b_{0}$ be the constant in Proposition 2.4.

(1) If (S-1) holds, then there exist constants $c_{1}>1$ and $b_{2}>0$ such that

[TABLE]

for all $(t,x)\in(0,T]\times(D\setminus\{0\})$.

(2) If (S-2) holds, then there exist a constant $c_{2}>1$ such that

[TABLE]

for all $(t,x)\in(0,T]\times D$ and $\eta>0$ where $a_{1}$ and $b_{1}$ are the constants in Lemma 2.7, and $\theta_{a}(r,t)=r\vee[\ell^{-1}(a/t)]^{-1}$ is defined as (2.14).

Before proving Theorem 1.5, we obtain a lower bound of $p_{D}(t,x,y)$ without (S-1) and (S-2). This result will be used later to obtain Green function estimates.

Proposition 5.1**.**

For every $T>0$, there exist positive constants $c_{1}=c_{1}(d,\psi,T,R_{0},\Lambda)$ and $c_{2}=c_{2}(d,\psi,T,R_{0},\Lambda)$ such that for all $(t,x,y)\in(0,T]\times(D\times D\setminus\text{diag})$,

[TABLE]

**Proof. **Let $R_{2}$ be the constant in Lemma 3.7. Fix $(t,x,y)\in(0,T]\times(D\times D\setminus\text{diag})$ and set

[TABLE]

Note that by (3.1), (3.2) and (3.3), we have $V(r_{t})\asymp t^{1/2}$ and $L(r_{t})\asymp h(r_{t})\asymp t^{-1}.$

Let $z_{x},z_{y}\in\partial D$ be the points satisfying $\delta_{D}(x)=|x-z_{x}|$ and $\delta_{D}(y)=|y-z_{y}|$. By (3.2), there exists a constant $m>1$ such that

[TABLE]

Case 1. Suppose that $|x-y|\leq R_{2}$. Define open neighborhoods of $x$ and $y$ as follows:

{\cal O}(x)=\begin{cases}B\big{(}x,V^{-1}[\frac{1}{8m}V(|x-y|)]\big{)},&\mbox{if }\;\;8mV(\delta_{D}(x))\geq V(|x-y|);\\ D\cap B(z_{x},\frac{1}{3}|x-y|),&\mbox{if }\;\;8mV(\delta_{D}(x))<V(|x-y|),\end{cases}

and

{\cal O}(y)=\begin{cases}B\big{(}y,V^{-1}[\frac{1}{8m}V(|x-y|)]\big{)},&\mbox{if }\;\;8mV(\delta_{D}(y))\geq V(|x-y|);\\ D\cap B\big{(}z_{y},\frac{1}{3}|x-y|\big{)},&\mbox{if }\;\;8mV(\delta_{D}(y))<V(|x-y|).\end{cases}

Then, we see that $x\in{\cal O}(x)\subset D,\,y\in{\cal O}(y)\subset D$ and

[TABLE]

Thus, by the strong Markov property and (5.1), we have (cf.   [17, Lemma 3.3],)

[TABLE]

To calculate the survival probability ${\mathbb{P}}_{x}(\tau_{{\cal O}(x)}>t)$, we first assume that $8mV(\delta_{D}(x))\geq V(|x-y|)$. In this case, we see from (4.1) and (3.1) that

[TABLE]

Next, assume that $8mV(\delta_{D}(x))<V(|x-y|)$. Note that by the monotonicity of $V$ and (5.5), we get $|x-y|>8\delta_{D}(x)$ in this case. Let $\rho:=V^{-1}(\varepsilon V(l_{t}(x,y)))$ where $\varepsilon\in(0,(8m)^{-1})$ will be chosen later. Then, we see from (3.1) and (3.2) that

[TABLE]

Note that we can not expect that $\rho\asymp l_{t}(x,y)$ in general.

If $8\delta_{D}(x)\geq\rho$, then by (4.1) and (5.8), we have

[TABLE]

Indeed, by Lemma 2.2(i) and (3.1), we see that $h(\rho/8)\asymp h(4\rho)$. Thus, if $l_{t}(x,y)=|x-y|/4$, then we get (5.9). Otherwise, if $l_{t}(x,y)=r_{t}$, then {\mathbb{P}}_{x}(\tau_{{\cal O}(x)}>t)\asymp 1\asymp\exp\big{(}-c_{3}th(|x-y|)\big{)} and hence (5.9) holds.

If $8\delta_{D}(x)<\rho$, then we can find a piece of annulus ${\cal A}(x)\subset\{w\in{\cal O}(x):\rho<|w-z_{x}|<|x-y|/4\}$ such that $\text{dist}({\cal A}(x),\partial{\cal O}(x))>\rho/8$. Recall that $\theta$ is shift operator. Then, by the strong Markov property, the Lévy system, (4.1), (3.8), (3.1) and (3.2), we have

[TABLE]

where $c_{5}>1$ is a constant independent of choice of $\varepsilon$. Now, we choose $\varepsilon=(2c_{5})^{-1}\wedge(16m)^{-1}$. Then, we get from (5.8) that

[TABLE]

Finally, by combining the above inequality with (5.7) and (5.9), we deduce that

[TABLE]

By the same way, we get {\mathbb{P}}_{y}(\tau_{{\cal O}(y)}>t)\geq c\big{(}1\wedge\frac{1}{tL(\delta_{D}(y))}\big{)}^{1/2}\exp\big{(}-c_{5}th(|x-y|)\big{)}. Then, (5) yields the desired lower bound.

Case 2. Suppose that $|x-y|>R_{2}$. In this case, we let $D_{x}:=D\cap B(x,R_{2}/4)$ and $D_{y}:=D\cap B(y,R_{2}/4)$. By the same argument as (5), (5.1) and Corollary 4.3, we get

[TABLE]

This completes the proof. $\Box$

Now, we are ready to prove Theorem 1.5.

Proof of Theorem 1.5. Fix $(t,x,y)\in(0,T]\times(D\times D\setminus\text{diag})$ and continue using the notation $r_{t}$ and $l_{t}(x,y)$ in (5.4).

(i) Since we have proved the lower bound in Proposition 5.1, it suffices to show that there exist $c_{1}>0,b_{3}\in(0,b_{0}]$ such that for all $(t,x,y)\in(0,T]\times(D\times D\setminus\text{diag})$,

[TABLE]

where $b_{0}$ is the constant in Proposition 2.4. Indeed, if (5.10) holds, then by the semigroup property and (5.2), we get

[TABLE]

which yields the result.

Now, we prove (5.10). If $\delta_{D}(x)\geq r_{t}/2$, then (5.10) is a consequence of (5.2) and the trivial bound that $p_{D}(t,x,y)\leq p(t,x-y)$. Hence, we assume that $\delta_{D}(x)<r_{t}/2$. By (3.3), there exists a constant $M>1$ such that

[TABLE]

Observe that by the semigroup property, monotonicity of $p(t,\cdot)$ and Proposition 4.2, we have

[TABLE]

Thus, if $ML(\delta_{D}(y))\geq L(\delta_{D}(x))$, then we get (5.10). Therefore, we assume that $ML(\delta_{D}(y))<L(\delta_{D}(x))$. Since $L$ is strictly decreasing, it follows from (5.11) that $\delta_{D}(y)>16\delta_{D}(x)$ and hence $|x-y|\geq|y-z_{x}|-|z_{x}-x|\geq\delta_{D}(y)-\delta_{D}(x)>15\delta_{D}(x)$ where $z_{x}\in\partial D$ is the point satisfying $\delta_{D}(x)=|x-z_{x}|$. Define

[TABLE]

and $W_{2}:=D\setminus(W_{1}\cup W_{3})=\{w\in D\setminus W_{1}:|w-y|>|x-y|/2\}$. Then, for $u\in W_{1}$ and $w\in W_{3}$, we obtain

[TABLE]

Observe that by the strong Markov property,

[TABLE]

First, by the Lévy system and (5.12), we get

[TABLE]

By (5.2) and Lemma 2.1, for all $s\in(0,T]$ and $l\in(0,2r_{t}]$, we have

[TABLE]

It follows that for all $s\in(0,t]$,

[TABLE]

Indeed, if $|x-y|>2r_{t}$, then we have $sh(|x-y|)\leq sh(2r_{t})\asymp st^{-1}\leq 1$. Moreover, by the semigroup property, Proposition 4.2, (5.15) and monotonicity of $h$, we get

[TABLE]

Then, using (5), (5.1), (5.15), (5) and Proposition 4.2, we obtain

[TABLE]

Second, by monotonicity of $p(t,\cdot)$, (5.2), (5.1) and Proposition 4.2, we get

[TABLE]

In the last inequality, we used (3.1), $L(r_{t})\asymp t^{-1}$ and the fact thats $e^{x}\geq 2e\sqrt{x}$ for all $x>0$ and $h(r)\geq L(r)$ for all $r>0$.

Lastly, we note that $t\mapsto te^{-at}$ is increasing on $(0,1/a)$ and decreasing on $(1/a,\infty)$. Thus, using similar calculation as the one given in (5), by monotonicity of $p(t,\cdot)$, (5.2), (5.1), Proposition 4.2 and (3.1), we have

[TABLE]

Combining the above inequality with (5), (5) and (5), we get (5.10).

(ii) We use the same notations as in the proof of (i) and follow the proof of (i).

(Upper bound) By the semigroup property and (5), it suffices to show that there exist positive constants $c_{1}$ and $b_{4}$ such that

[TABLE]

By the similar argument to the one given in the proof of (i), we may assume that $\delta_{D}(x)<r_{t}/2$. Moreover, observe that for every $u,v\in D$, by the triangle inequality, $\max\{|x-u|,|u-v|,|v-y|\}\geq|x-y|/3$. Thus, by the semigroup property, monotonicity of $p(t,\cdot)$ and Proposition 4.2, we have that

[TABLE]

Therefore, if $ML(\delta_{D}(y))\leq L(\delta_{D}(x))$ for some $M>0$, then we get (5.20) from (5). Hence, we may assume that $\delta_{D}(y)>16\delta_{D}(x)$ by the same argument as the one given in the proof of (i).

To prove (5.20), we first assume that $|x-y|\leq[\ell^{-1}(3a_{1}/t)]^{-1}$. In this case, we have that $\theta_{a_{1}}(|x-y|,t/3)=\theta_{3a_{1}}(|x-y|,t)=[\ell^{-1}(3a_{1}/t)]^{-1}$. Then, by the semigroup property, (5) and Proposition 4.2, we get

[TABLE]

Now, suppose that $|x-y|>[\ell^{-1}(3a_{1}/t)]^{-1}$. In this case, we use (5) and find upper bounds for $I_{1}$, $I_{2}$ and $I_{3}$. Observe that for all $s\in(0,T]$ and $l\in(0,2r_{t}]$, by (5) and the similar calculation to the one given in (5.15),

[TABLE]

Then, by using (5) instead of (5.15), we have that for all $0<s\leq T$,

[TABLE]

Hence, by the similar arguments to the ones given in (5) and (5), we obtain

[TABLE]

Next, by (5), (5.1), monotonicity of $h$, we have

[TABLE]

Let f(r):=r^{-d}\exp\big{(}-c_{7}th(r)\big{)} where the constant $c_{7}\in(0,b_{1}/3)$ will be chosen later. Then, by (2.2), there exists a constant $c_{6}>0$ such that for $r\in(0,[\ell^{-1}(a_{1}/t)]^{-1})$,

[TABLE]

Set $c_{7}=d/(3a_{1}c_{6})\wedge b_{1}/3$. Then, we see that $f$ is decreasing on \big{(}[\ell^{-1}(3a_{1}/t)]^{-1},[\ell^{-1}(a_{1}/t)]^{-1}\big{)}. Using this fact, since $\ell$ is almost increasing, we deduce that

[TABLE]

It follows that by the same argument as in the one given in (5),

[TABLE]

Lastly, we note that since $|x-y|>[\ell^{-1}(3a_{1}/t)]^{-1}$,

[TABLE]

Therefore, by the same proof as in the one given in (i), we obtain

[TABLE]

This finishes the proof for the upper bound.

(Lower bound) Fix $\eta>0$. By Proposition 5.1, it remains to prove the lower bound when $|x-y|<[\ell^{-1}(\eta/t)]^{-1}\wedge R_{2}$, where $R_{2}$ is the constant in Lemma 3.7. Let $\zeta_{t}:=[\ell^{-1}(\eta/t)]^{-1}\wedge R_{2}$ and define open neighborhoods of $x$ and $y$ as follows. Recall that $z_{x},z_{y}\in\partial D$ are the points satisfying $\delta_{D}(x)=|x-z_{x}|$ and $\delta_{D}(y)=|y-z_{y}|$. We define

{\cal U}(x)=\begin{cases}B\big{(}x,V^{-1}(\frac{1}{8m}V(\zeta_{t}))\big{)},&\mbox{if}\;\;8mV(\delta_{D}(x))\geq V(\zeta_{t});\\[2.0pt] B(z_{x},\frac{1}{3}\zeta_{t})\cap D,&\mbox{if}\;\;8mV(\delta_{D}(x))<V(\zeta_{t}),\end{cases}

and

{\cal U}(y)=\begin{cases}B\big{(}y,V^{-1}(\frac{1}{8m}V(\zeta_{t}))\big{)},&\mbox{if }\;\;8mV(\delta_{D}(y))\geq V(\zeta_{t});\\[2.0pt] B(z_{y},\frac{1}{3}\zeta_{t})\cap D,&\mbox{if }\;\;8mV(\delta_{D}(y))<V(\zeta_{t}),\end{cases}

where $m$ is the constants in (5.5). Then, we can see that $x\in{\cal U}(x)\subset D$ and $y\in{\cal U}(y)\subset D$.

We claim that there exist a constant $c_{4}>0$ independent of the choice of $\eta$, and a constant $c_{3}>0$ such that

[TABLE]

Indeed, if $8mV(\delta_{D}(x))\geq V(\zeta_{t})$, then by (4.1) and (3.1), we have

[TABLE]

Suppose that $8mV(\delta_{D}(x))<V(\zeta_{t})$. If $\zeta_{t}=R_{2}$, then by Corollary 4.3, we get

[TABLE]

Otherwise, if $\zeta_{t}=[\ell^{-1}(\eta/t)]^{-1}<R_{2}$, then by the similar argument to the one given in the proof of Proposition 5.1,

[TABLE]

In the second inequality, we used $L(\zeta_{t})\geq cK(\zeta_{t})\asymp\ell(\zeta_{t}^{-1})\asymp t^{-1}$ which follows from the proof of Lemma 2.1 and (2.2). By combining (5.23), (5.24) and (5), we obtain (5.22).

Let $w_{x}:=z_{x}+4\zeta_{t}(x-z_{x})/|x-z_{x}|\in D$ and define

[TABLE]

Then, for all $u\in{\cal U}(x)$ and $v\in{\cal W}$, we have $|u-v|\asymp\zeta_{t}$. Moreover, since $|x-y|<\zeta_{t}$, we also have $|u^{\prime}-v|\asymp\zeta_{t}$ for all $u^{\prime}\in{\cal U}(y)$ and $v\in{\cal W}$. Thus, for every $v\in{\cal W}_{0}$, by the similar argument to (5), (4.1) and (5.22), we have

[TABLE]

Similarly, we also have that

[TABLE]

It follows that by the semigroup property and (A),

[TABLE]

If $\zeta_{t}=[\ell^{-1}(\eta/t)]^{-1}$, then since $\ell$ is almost increasing, we get $\ell(\zeta_{t}^{-1})\asymp t^{-1}$. Hence, we are done. Otherwise, if $\zeta_{t}=R_{2}$, then we have $t\asymp 1$ and hence t^{2}\ell(\zeta_{t}^{-1})\nu(\zeta_{t})\exp\big{(}-2c_{5}th(\zeta_{t})\big{)}\asymp t\nu([\ell^{-1}(\eta/t)]^{-1})\exp\big{(}-cth([\ell^{-1}(\eta/t)]^{-1})\big{)}\asymp 1. This completes the proof. $\Box$

6. Large time estimates

In this section, we give the proof of Theorem 1.6. Let $D$ be a fixed bounded $C^{1,1}$ open subset in ${\mathbb{R}}^{d}$ of scale $(r_{1},r_{2})$ and $x_{1},x_{2}\in{\mathbb{R}}^{d}$ be the fixed points satisfying $B(x_{1},r_{1})\subset D\subset B(x_{2},r_{2})$. We mention that under condition (L-1), the transition semigroup $\{P_{t}^{D},t\geq 0\}$ of $Y_{t}^{D}$ may not be compact operators in $L^{2}(D;dx)$, though $D$ is bounded. (See, Proposition 2.3.) Hence, in that case, we need some lemmas to obtain the large time estimates instead of the general spectral theory.

Lemma 6.1**.**

There exists a constant $c_{1}>0$ which only depend on the dimension $d$ such that for all $(t,x,y)\in(0,\infty)\times(D\times D\setminus\text{diag})$,

[TABLE]

**Proof. **By the semigroup property, we have

[TABLE]

Hence, we obtain the result from (4.1). $\Box$

Define for $r\geq 1$,

[TABLE]

Note that if (L-1) holds, we have that

[TABLE]

Moreover, by the same argument as the one given in the proof of Lemma 2.2, there exist positive constants $C_{6}$ and $C_{7}$ which only depend on the dimension $d$ and $\kappa_{1}$ and $\kappa_{2}$ in (1.3) such that

[TABLE]

We also note that ${\widehat{\Phi}}$ satisfies $\text{WS}^{\infty}(0,2,1)$. Here, we get the large time on-diagonal estimates for $p(t,x)$ under condition (L-1).

Lemma 6.2**.**

Assume that (L-1) holds. Then, there exists a positive constant $b_{5}=b_{5}(d,\psi,r_{2})$ such that for any $T>0$, there exist $c_{1},c_{2}>0$ such that for all $t\in[T,\infty)$ and $|x|\leq 2r_{2}$,

[TABLE]

**Proof. **Fix $x\in{\mathbb{R}}^{d}$ satisfying $|x|\leq 2r_{2}$ and let $r=|x|$. By [25, (5.4)], the mean value theorem, Lemma 2.6, (6.1) and (6.2), we have that, for all $t>0$ (cf. (2)),

[TABLE]

First, by (6.1), and the monotonicity and the scaling properties of ${\widehat{\ell}}$, $\ell$ and ${\widehat{\Phi}}$, we have

[TABLE]

In the last inequality above, we used the facts that $e^{x}\geq x$ for $x>0$ and ${\widehat{\Phi}}(1/r)\geq{\widehat{\Phi}}(1/(2r_{2}))$.

On the other hand, set q_{\gamma,k}(u)=q_{\gamma,k}(u,t):=u^{\gamma}\exp\big{(}-kt{\widehat{\Phi}}(u)\big{)} for $u\geq 2$ and $\gamma,k>0$. We observe that for any $\gamma,k>0$,

[TABLE]

Since ${\widehat{\ell}}$ is increasing, it follows that there exists $u_{0}\in[2,\infty)$ such that $q$ is decreasing on $[2,u_{0}]$ and increasing on $[u_{0},\infty)$. Thus, for any $[a,b]\subset[2,\infty)$ and $\gamma,k>0$, we have that

[TABLE]

Choose any constant $\varepsilon\in(0,1)$ such that $(1-\varepsilon)d+\alpha_{1}>0$. This is possible since $\alpha_{1}>-d$. Then we set $\rho:=((1-\varepsilon)d+\alpha_{1})/(1+\varepsilon)\in(0,d+\alpha_{1})$ so that

[TABLE]

First, suppose that $q_{d+\rho,\,C_{6}}(2)\geq q_{d+\rho,\,C_{6}}(4r_{2}/r)$. Then by the change of variables and (6.6), since ${\widehat{\ell}}$ is increasing, we have

[TABLE]

Hence, we obtain (6.3) from (6), (6), (1.3) and (6.2) in this case.

Now, we suppose that $q_{d+\rho,\,C_{6}}(2)<q_{d+\rho,\,C_{6}}(4r_{2}/r)$. Note that by (6.1) and the scaling property of $\ell$, it holds that for any $2<u\leq s$,

[TABLE]

We also note that for any $\gamma,k>0$ and $u\geq 2$, the map $q_{\gamma,k}(u)\leq q_{\gamma,\varepsilon k}(u)$ and $q_{\gamma,k}(u)^{s}=q_{s\gamma,\,sk}(u)$ for all $s>0$. Thus, by the change of variables, (6.1), (6.6), (6.7) and the scaling properties of $\ell$ and ${\widehat{\Phi}}$, since we have assumed $q_{d+\rho,\,C_{6}}(2)<q_{d+\rho,\,C_{6}}(4r_{2}/r)$, we get that

[TABLE]

Then we get (6.3) by using (6), (6), (1.3) and (6.2) again. $\Box$

Now, we give the proof of Theorem 1.6.

Proof of Theorem 1.6. Let $a(x,y):=L(\delta_{D}(x))^{-1/2}L(\delta_{D}(y))^{-1/2}$.

(i) Choose $(t,x,y)\in[T,\infty)\times(D\times D\setminus\text{diag})$ and let $x_{1}\in D$ be the point satisfying $B(x_{1},r_{1})\subset D$. By the semigroup property, Theorem 1.5(i), (5.2) and (4.1), we have

[TABLE]

On the other hand, since $D$ is a bounded set, one can follow the proof of Proposition 5.1, after changing the definition of $l_{t}(x,y)$ therein from $r_{t}\wedge(|x-y|/4)$ to $|x-y|/4$, and see that

[TABLE]

In the last inequality above, we used the fact that $L(2r_{2})/L(\delta_{D}(z))\leq L(2r_{2})/L(r_{2})\leq c$ for all $z\in D$, which comes from the monotonicity of $L$ and (3.3). By combining (6) with (6), we get the desired lower bound.

Now, we prove the upper bound. By the semigroup property, Theorem 1.5(i), Corollary 2.13, Lemma 6.1 and Lemma 6.2, we get

[TABLE]

which yields the upper bound.

(ii) & (iii) Since the proof of (iii) is similar and easier, we only provide the proof of (ii). By Proposition 2.3, there exist $T_{0}>0$ such that the transition semigroup $\{P_{t}^{D}:t\geq T_{0}\}$ of $Y_{t}^{D}$ consists of compact operators. Let $0<\mu_{1}<1$ be the largest eigenvalue of the operator $P_{T_{0}}^{D}$ and $\phi_{1}\in L^{2}(D;dx)$ be the corresponding eigenfunction with unit $L^{2}$-norm. For $n\geq 1$, we denote by $\{\mu_{n,k};k\geq 1\}\subset(0,1)$ the discrete spectrum of $P_{nT_{0}}^{D}$, arranged in decreasing order and repeated according to their multiplicity and $\{\phi_{n,k};k\geq 1\}$ be the corresponding eigenfunctions with unit $L^{2}$-norm. Then, by the semigroup property, we have $\mu_{n,1}=\mu_{1}^{n}$ and $\phi_{n,1}=\phi_{1}$ for all $n\geq 1$. From the eigenfunction expansion of $p_{D}(nT_{0},x,\cdot)$ and Parseval’s identity, we have for $n\geq 1$,

[TABLE]

On the other hand, for all $s>0$ and $x\in D$, since $p(T_{0},0)\leq c_{0}<\infty$, we have

[TABLE]

Thus, we obtain for all $0<s\leq T_{0}$ and $n\geq 1$,

[TABLE]

For $t\geq 4T_{0}$ and $x,y\in D$, we let $n:=\lfloor(t-3T_{0})/T_{0}\rfloor\geq 1$ and $s:=t-(n+2)T_{0}\in[T_{0},2T_{0})$. Recall $a(x,y)=L(\delta_{D}(x))^{-1/2}L(\delta_{D}(y))^{-1/2}$. By (6.10) and Corollary 4.3, we have

[TABLE]

where $\lambda_{1}:=T_{0}^{-1}\log(\mu_{1}^{-1})$. Moreover, by Theorem 1.5, Corollary 4.3 and (6), we get

[TABLE]

This completes the proof. $\Box$

7. Green function estimates

In this section, we provide the proof of Theorem 1.7. Throughout this section, we assume that (D) holds.

Lemma 7.1**.**

For all Borel set $D$ and $x,y\in D$, we have

[TABLE]

In particular, if $D$ is bounded then for all $x,y\in D$

[TABLE]

**Proof. **Since $(1\wedge p)(1\wedge q)\leq 1\wedge(pq)$ for every $p,q>0$, one inequality in (7.1) is trivial. On the other hand, since $1\wedge(p/q)\asymp p/(p+q)$ for every $p,q>0$, it suffices to prove that

[TABLE]

By symmetry, we may assume that $\delta_{D}(x)\leq\delta_{D}(y)$. According to the subadditivity of $V$,

[TABLE]

This proves (7.1). The second claim follows from (3.1). $\Box$

Lemma 7.2**.**

It holds that

[TABLE]

**Proof. **Since the Lévy measure $\nu$ is infinite, we have $\lim_{r\to 0}L(r)=\infty$. Thus, it suffices to show that the second equality holds. By l’Hospital’s rule, [13, Lemmas 3.1 and 3.2], (1.3) and (3.3), since $\alpha_{2}<d$, we have

[TABLE]

Indeed, since $r\mapsto\nu(r^{-1})$ satisfies $\text{WS}^{\infty}(d+\alpha_{1},d+\alpha_{2},1)$ and $d+\alpha_{1}>0$, according to [13, Lemma 3.1 and 3.2], there exists a function $\widetilde{\nu}(r)$ such that for all $0<r<1$, $\widetilde{\nu}(r)\asymp\nu(r)$ and $-\widetilde{\nu}^{\prime}(r)\asymp r^{-1}\widetilde{\nu}(r)\asymp r^{-1}\nu(r)$. Hence, the first inequality in the display holds. $\Box$

Recall that for a Borel subset $D\subset{\mathbb{R}}^{d}$, the Green function $G_{D}(x,y)$ is defined by

[TABLE]

Since the process $Y$ can be recurrent, we can not expect to obtain upper estimates for $G_{{\mathbb{R}}^{d}}(x,y)$ in general. However, when $D$ is bounded, we can establish a prior upper estimates for $G_{D}(x,y)$ regardless of transience of $Y$. By $\text{diam}(D)$ we denote the diameter of $D$.

Lemma 7.3**.**

Let $D\subset{\mathbb{R}}^{d}$ be a bounded Borel set. Then, there exists a constant $c_{1}=c_{1}(d,\psi,\text{diam}(D))>0$ such that for all $x,y\in D$,

[TABLE]

**Proof. **Fix $x,y\in D$ and let $r=|x-y|$. If $x=y$, by Lemma 7.2, there is nothing to prove. Hence, we assume that $r>0$.

By Lemma 6.1, (2) and Fubini’s Theorem, we have

[TABLE]

First, by the change of the variables, we have

[TABLE]

Observe that by applying l’Hospital’s rule three times and the same argument as the one, coming from [13, Lemmas 3.1 and 3.2], given in the proof of Lemma 7.2, we obtain

[TABLE]

The fourth inequality above is valid, since we can assume that $-(r^{-d}/\ell(r^{-1}))^{\prime}\asymp r^{-d-1}/\ell(r^{-1})$ for $0<r<1$ by the argument given in the proof of Lemma 7.2 because $r\mapsto r^{d}/\ell(r)$ satisfies $\text{WS}^{\infty}(d-\alpha_{2},d-\alpha_{1},1)$ and $\alpha_{2}<d$. In the third and fifth inequalities above, we used the fact that $\ell(r^{-1})\leq cL(r)$ for $0<r<1$, which follows from (2.2) and the proof of Lemma 2.1. Thus, since $D$ is bounded, we get that $I_{1}\leq c\ell(r^{-1})L(r)^{-2}$.

On the other hand, by the scaling property of $\ell$ and the monotonicity of $\Phi$, we obtain

[TABLE]

This completes the proof. $\Box$

Proof of Theorem 1.7. Fix $x,y\in D$ and set $a(x,y):=L(\delta_{D}(x))^{-1/2}L(\delta_{D}(y))^{-1/2}$. By (1.3) and Lemma 2.1, it suffices to prove that

[TABLE]

(Lower bound) By Proposition 5.1, we have that

[TABLE]

In the above, we used the change of the variables $s=th(|x-y|)$ in the third line, the fact that $h(r)\geq L(r)$ for all $r>0$ in the fourth line, and Lemma 7.1 and (3.1) in the fifth line.

(Upper bound) Using boundary Harnack principle and Lemma 7.3, one can prove the upper bound following the proofs of [29, Theorem 1.2 and Theorem 6.4] and [32, Theorem 4.6] line by line. Thus, we provide the main steps of the proof only.

By the boundary Harnack principle (see, [23, Theorem 1.9]), Lemma 7.3 and (7), we can follow the proof of [29, Theorem 6.4] to obtain

[TABLE]

where $g_{D}(z):=G_{D}(z,z_{0})\wedge c_{1}$ for some fixed constant $c_{1}>0$, $z_{0}\in D$ is a fixed point in $D$ and $A\in{\cal B}(x,y)$, where ${\cal B}(x,y)$ is given by [29, (6.7)]. Moreover, we can also follow the proof of [32, Theorem 4.6] to show that for all $z\in D$,

[TABLE]

Indeed, let $R_{3}:=\delta_{D}(z_{0})\wedge R_{2}$ where $R_{2}$ is the constant in Lemma 3.7. If $\delta_{D}(z)\geq R_{3}/8$, then we get $L(\delta_{D}(z))^{-1/2}\geq L(R_{3}/8)^{-1/2}\geq c_{1}^{-1}L(R_{3}/8)^{-1/2}g_{D}(z)$. Moreover, by (7) and Lemma 7.2, we also get $g_{D}(z)\geq c\geq cL(r_{2})^{-1/2}\geq cL(\delta_{D}(z))^{-1/2}$. Hence, (7.4) holds in this case.

Next, we assume that $\delta_{D}(z)<R_{3}/8$. Then, we get that $|z-z_{0}|\geq\delta_{D}(z_{0})-\delta_{D}(z)\geq 7R_{3}/8$. Therefore, by Lemma 7.2, $g_{D}(z)\asymp G_{D}(z,z_{0})$. Choose $w_{z}\in\partial D$ satisfying $\delta_{D}(z)=|z-w_{z}|$. Let $z^{*}:=w_{z}+R_{3}(z-w_{z})/(4|z-w_{z}|)\in D$ and define $U(z,1)$ as (4.3). Then, by the boundary Harnack principle, (3.9), Lemma 7.3, (7) and Proposition 4.2, we get

[TABLE]

Hence, we obtain (7.4).

We see from the definition of ${\cal B}(x,y)$ that $\delta_{D}(A)\geq c|x-y|$ for some constant $c>0$. Thus, by combining (7.3) and (7.4), we get from (3.3) that

[TABLE]

This together with Lemma 7.3 completes the proof. $\Box$

8. Example

In this section, we give an example that is covered by our results.

Example 8.1**.**

Let $Y=(Y_{t}:t\geq 0)$ be a pure jump isotropic unimodal Lévy process with Lévy measure $\nu$ satisfying (A) and (B), and $D$ be a $C^{1,1}$ open set in ${\mathbb{R}}^{d}$ with characteristics $(R_{0},\Lambda)$. Suppose that there exists $p\in[-1,\infty)$ such that

[TABLE]

Typical examples of isotropic unimodal Lévy processes satisfying (8.1) are geometric stable processes $(p=0)$ and iterated geometric stable processes $(p=-1)$. The condition $p\geq-1$ is necessary to make the Lévy measure $\nu$ be infinite. We let

[TABLE]

Then, for every fixed $R>0$, we have that for $0<r\leq R$,

[TABLE]

We first obtain the small time estimates for the Dirichlet heat kernel. Define for $p>-1$,

[TABLE]

and

[TABLE]

(Case 1) $p>0.$

In this case, (S-2) holds. Note that we do not need the condition (C) when we estimate $p_{D}(t,x,y)$ only for $|x-y|\leq 1$. Thus, according to Theorem 1.5(ii), for every $T>0$, there are constants $c_{0}>1$, $c_{1},...,c_{6}>0$ such that for all $(t,x,y)\in(0,T]\times D\times D$ satisfying $|x-y|\leq 1$,

[TABLE]

where

[TABLE]

(Case 2) $-1<p\leq 0.$

Since (S-1) holds, by Theorem 1.5(i), for every $T>0$, there are constants $c_{0}>1$, $c_{1},c_{2}>0$ such that for all $(t,x,y)\in(0,T]\times(D\times D\setminus\text{diag})$ satisfying $|x-y|\leq 1$,

[TABLE]

(Case 3) $p=-1.$

Since (S-1) holds, by Theorem 1.5(i), for every $T>0$, there are constants $c_{0}>1$, $c_{1},c_{2}>0$ such that for all $(t,x,y)\in(0,T]\times(D\times D\setminus\text{diag})$ satisfying $|x-y|\leq 1$,

[TABLE]

Now, we further assume that $D$ is bounded and of scale $(r_{1},r_{2})$. Then, we get the following large time estimates.

(Case 1) $p\geq 0.$

Since either (S-2) or (L-2) holds, by Theorem 1.6(ii, iii), there exist $T_{1}\geq 0$ (if $p>0$, then $T_{1}=0$) and $\lambda_{1}>0$ such that for every $T>T_{1}$, we have that for all $(t,x,y)\in[T,\infty)\times D\times D$,

[TABLE]

(Case 2) $-1<p<0$.

Since (L-1) holds, by Theorem 1.6(i), for every $T>0$, there are constants $c_{0},c_{1},c_{2},\lambda_{2},\lambda_{3}>0$ such that for all $(t,x,y)\in[T,\infty)\times(D\times D\setminus\text{diag})$,

[TABLE]

(Case 3) $p=-1$.

Since (L-1) holds, by Theorem 1.6(i), for every $T>0$, there are constants $c_{0},c_{1},c_{2},\lambda_{2},\lambda_{3}>0$ such that for all $(t,x,y)\in[T,\infty)\times(D\times D\setminus\text{diag})$,

[TABLE]

Finally, we obtain the Green function estimates by Theorem 1.7. Let $D$ be a bounded $C^{1,1}$ open set in ${\mathbb{R}}^{d}$. Note that the Lévy measure $\nu$ satisfies (D). Hence, we have that, for all $x,y\in D$, if $p>-1$, then

[TABLE]

and if $p=-1$, then

[TABLE]