Asymptotics of maximum likelihood estimation for stable law with continuous parameterization

TL;DR

This paper investigates the asymptotic properties of maximum likelihood estimation for alpha-stable laws using a continuous parameterization, addressing discontinuities in traditional parameterizations and providing a more practical theoretical framework.

Contribution

It introduces a continuous parameterization for alpha-stable laws, filling gaps in existing asymptotic theory and enabling accurate Fisher information approximation near the Cauchy law.

Findings

Asymptotic normality and consistency are established for the continuous parameterization.

Fisher information matrix is numerically approximated and shown to be continuous at the Cauchy law.

Discontinuities at certain parameter points are resolved, improving practical applicability.

Abstract

Asymptotics of maximum likelihood estimation for $\alpha$-stable law are analytically investigated with a continuous parameterization. The consistency and asymptotic normality are shown on the interior of the whole parameter space. Although these asymptotics have been provided with Zolotarev's $(B)$ parameterization, there are several gaps between. Especially in the latter, the density, so that scores and their derivatives are discontinuous at $\alpha=1$ for $\beta\neq 0$ and usual asymptotics are impossible. This is considerable inconvenience for applications. By showing that these quantities are smooth in the continuous form, we fill gaps between and provide a convenient theory. We numerically approximate the Fisher information matrix around the Cauchy law $(\alpha,\beta)=(1,0)$. The results exhibit continuity at $\alpha=1,\,\beta\neq 0$ and this secures the accuracy of our calculations.

Tables

Table 1. Table 1. Fisher information matrix at Cauchy ( α , β ) = ( 1 , 0 ) 𝛼 𝛽 1 0 (\alpha,\beta)=(1,0)

$I_{\alpha \alpha }$	$I_{\beta \beta }$	$I_{\sigma \sigma }$	$I_{\mu \mu }$	$I_{\alpha \beta }$	$I_{\alpha \sigma }$	$I_{\alpha \mu }$	$I_{\beta \sigma }$	$I_{\beta \mu }$	$I_{\sigma \mu }$
0.859	0.348	0.5	0.5	0	-0.135	0	0	0.086	0

Table 2. Table 2. Approximated Fisher informations for α 𝛼 \alpha and β 𝛽 \beta around Cauchy

	${\tilde{I}}_{\alpha \alpha }$					${\tilde{I}}_{\beta \beta }$
$\alpha \setminus \beta$	$0.1$	$0.05$	$0.01$	$0.001$	$0$	$0.1$	$0.05$	$0.01$	$0.001$	$0$
0.95	1.096	1.087	1.084	1.084	1.084	0.392	0.392	0.391	0.391	0.391
0.99	0.907	0.9	0.898	0.897	0.897	0.357	0.356	0.356	0.356	0.356
0.999	0.872	0.865	0.863	0.863	0.863	0.349	0.349	0.349	0.349	0.349
1	0.874	0.864	0.86	0.859	0.859	0.349	0.348	0.348	0.348	0.348
1.001	0.865	0.858	0.855	0.855	0.855	0.348	0.347	0.347	0.347	0.347
1.01	0.832	0.825	0.823	0.823	0.823	0.341	0.34	0.34	0.34	0.34
1.05	0.71	0.704	0.702	0.702	0.702	0.312	0.311	3.11	3.11	3.11

Equations

$$\displaystyle\begin{split}\varphi(t)=\left\{\begin{array}[]{ll}\exp\Big{(}-|\sigma t|^{\alpha}\big{\{}1+i\beta\,{\rm sign}t\tan\frac{\pi\alpha}{2}(|\sigma t|^{1-\alpha}-1)\big{\}}+i\mu t\Big{)}&\text{if}\ \alpha\neq 1\\ \exp\Big{(}-|\sigma t|-i\sigma t\,(2\beta/\pi)\log|\sigma t|+i\mu t\Big{)}&\text{if}\ \alpha=1,\end{array}\right.\end{split}$$

$$f(x;\mu ,\sigma ,\alpha ,\beta )$$

$$\displaystyle=\frac{1}{\sigma}\frac{1}{\pi}\mathrm{Re}\int_{0}^{\infty}e^{-it\big{(}\frac{x-\mu}{\sigma}+\beta\tan\frac{\pi\alpha}{2}\big{)}-|t|^{\alpha}\big{(}1-i\beta\tan\frac{\pi\alpha}{2}\big{)}}dt$$

$$=\frac{1}{\sigma }f(\frac{x-\mu }{\sigma };0,1,\alpha ,\beta ).$$

$$f_{{\theta }_i}^{\prime }=\frac{{\partial }^{2}f}{\partial x\partial {\theta }_i}=\frac{{\partial }^{2}f}{\partial {\theta }_i\partial x},f_{{\theta }_i{\theta }_j}=\frac{{\partial }^{2}f}{\partial {\theta }_i\partial {\theta }_j}=\frac{{\partial }^{2}f}{\partial {\theta }_j\partial {\theta }_i},i,j=1,\dots ,4,$$

$$\displaystyle\begin{split}\begin{array}[]{lll}f=O(|x|^{-(1+\alpha)}),&f_{\mu}^{\prime}=-f_{\mu\mu}=O(|x|^{-(3+\alpha)}),&f_{\sigma\alpha}=O(|x|^{-(1+\alpha)}\log|x|),\\ f_{\mu}=-f^{\prime}=O(|x|^{-(2+\alpha)}),&f_{\sigma}^{\prime}=-f_{\mu\sigma}=O(|x|^{-(2+\alpha)}),&f_{\sigma\beta}=O(|x|^{-(1+\alpha)}),\\ f_{\sigma}=O(|x|^{-(1+\alpha)}),&f_{\alpha}^{\prime}=-f_{\mu\alpha}=O(|x|^{-(2+\alpha)}\log|x|),&f_{\alpha\alpha}=O(|x|^{-(1+\alpha)}\log^{2}|x|),\\ f_{\alpha}=O(|x|^{-(1+\alpha)}\log|x|),&f_{\beta}^{\prime}=-f_{\mu\beta}=O(|x|^{-(2+\alpha)}),&f_{\alpha\beta}=O(|x|^{-(1+\alpha)}\log|x|),\\ f_{\beta}=O(|x|^{-(1+\alpha)}),&f_{\sigma\sigma}=O(|x|^{-(1+\alpha)}),&f_{\beta\beta}=O(|x|^{-(1+\alpha)}).\end{array}\end{split}$$

$$f_{\sigma \sigma }=O(|x{|}^{-3}\log |x|),f_{\beta \beta }=O(|x{|}^{-3}\log |x|).$$

$$f_{{\theta }_i{\theta }_j}(x;\theta )=\frac{1}{2\pi }{\int }_{-\infty }^{\infty }{e}^{-itx}{\phi }_{{\theta }_i{\theta }_j}(t)dt,1\le i,j\le 4,$$

$$f_{{\theta }_i}^{\prime }(x;\theta )=\frac{1}{2\pi }{\int }_{-\infty }^{\infty }it{e}^{-itx}{\phi }_{{\theta }_i}(t)dt,1\le i\le 4,$$

$$f_{\theta_{i}}(x;\theta)-f_{\theta_{i}}(y;\theta^{\prime})=\frac{1}{2\pi}\int_{-\infty}^{\infty}\big{\{}e^{-itx}(1-e^{it(x-y)})\varphi_{\theta}(t)+e^{-ity}(\varphi_{\theta}(t)-\varphi_{\theta^{\prime}}(t))\big{\}}dt,$$

$$f_{{\theta }_i}(x)$$

$$=\mathrm{Im}\frac{1}{\pi }{\int }_0^{\infty }{\psi }_{{\theta }_i}(t/i)e^{-tx+i(1+\beta )t-2\beta /\pi t\log t}dt$$

$$=\mathrm{Im}\frac{1}{\pi x}{\int }_0^{\infty }{\psi }_{{\theta }_i}(t/(ix))e^{-t+i(t/x)(1+\beta )-2\beta /\pi (t/x)\log (t/x)}dt.$$

$$\exp\Big{(}i\frac{t}{x}(1+\beta)-2\frac{\beta}{\pi}\frac{t}{x}\log\frac{t}{x}\Big{)}=\sum_{k=0}^{\infty}\frac{1}{k!}\Big{\{}i\frac{t}{x}(1+\beta)-\frac{2\beta}{\pi}\frac{t}{x}\log\frac{t}{x}\Big{\}}^{k},$$

$$f_{{\theta }_i}(x)$$

$$\ell (x;\theta )=\log f(x;\theta )\mathrm{and}{\ell }_{{\theta }_i}(x;\theta )=\frac{\partial \ell (x;\theta )}{\partial {\theta }_i}=\frac{f_{{\theta }_i}(x;\theta )}{f(x;\theta )},1\le i\le 4,$$

$${\ell }_{{\theta }_i}^{\prime }(x)$$

$${\ell }_{{\theta }_i{\theta }_j}(x)$$

$${\ell }_{{\theta }_i}(x),{\ell }_{{\theta }_i}^{\prime }(x)\text{and}{\ell }_{{\theta }_i{\theta }_j}(x),i,j=1,\dots ,4,$$

$$\displaystyle\begin{split}\begin{array}[]{ll}\ell_{\mu}(x)=O(|x|^{-1}),&\ell_{\mu\mu}(x)=-\ell_{\mu}^{\prime}(x)=O(|x|^{-2}),\\ \ell_{\sigma}(x)=O(1),&\ell_{\mu\sigma}(x)=-\ell_{\sigma}^{\prime}(x)=O(|x|^{-1}),\\ \ell_{\alpha}(x)=O(\log|x|),&\ell_{\mu\alpha}(x)=-\ell_{\alpha}^{\prime}(x)=O(|x|^{-1}\log|x|),\\ \ell_{\beta}(x)=O(1),&\ell_{\mu\beta}(x)=-\ell_{\beta}^{\prime}(x)=O(|x|^{-1}),\end{array}\end{split}$$

$$\displaystyle\begin{split}\begin{array}[]{lll}\ell_{\sigma\sigma}(x)=O(1),&\ell_{\sigma\alpha}(x)=O(\log|x|),&\ell_{\sigma\beta}(x)=O(1),\\ \ell_{\alpha\alpha}(x)=O(\log^{2}|x|),&\ell_{\alpha\beta}(x)=O(\log|x|),&\ell_{\beta\beta}(x)=O(1).\end{array}\end{split}$$

$$\theta \mapsto L_n(\theta )=\frac{1}{n}\sum _{k=1}^n\ell (X_k;\theta ),$$

$$\sqrt{n}({\hat{\theta }}_n-{\theta }_0)=I_{{\theta }_0}^{-1}\frac{1}{\sqrt{n}}\sum _{k=1}^n{\ell }_{{\theta }_0}(X_k)+o_{P_{{\theta }_0}}(1),$$

$$\underset{\theta \in C}{\sup }|L_n(\theta )-L(\theta )|\stackrel{p}{\to }0,$$

$$\underset{\theta :d(\theta ,{\theta }_0)\ge \epsilon ,\theta \in C}{\sup }L(\theta )<L({\theta }_0),$$

$$|\log p_{{\theta }^{\prime }}(x)-\log p_{{\theta }^{\prime \prime }}(x)|\le \dot{\eta }(x)\Vert {\theta }^{\prime }-{\theta }^{\prime \prime }\Vert ,$$

$$\sqrt{n}({\hat{\theta }}_n-{\theta }_0)=I_{{\theta }_0}^{-1}\frac{1}{\sqrt{n}}\sum _{k=1}^n{\ell }_{{\theta }_0}(X_k)+o_{P_{{\theta }_0}}(1).$$

$$I_{{\theta }_i{\theta }_j}=\int {\ell }_{{\theta }_i}(x){\ell }_{{\theta }_j}(x)/f(x;\theta )dx,i,j=1,2,3,4.$$

$$\displaystyle\big{|}\log p_{\theta^{\prime}}(x)-\log p_{\theta^{\prime\prime}}(x)\big{|}\leq\big{|}\sum_{i=1}^{4}\ell_{\theta_{i}}(x;\theta_{i}^{\ast})(\theta_{i}^{\prime}-\theta_{i}^{\prime\prime})\big{|}\leq\sum_{i=1}^{4}|\ell_{\theta_{i}}(x;\theta_{i}^{\ast})|\cdot\|\theta^{\prime}-\theta^{\prime\prime}\|,$$

$${\mathbf{a}}^{\prime }{\ell }_{\theta }(x)=\frac{1}{2\pi f(x;\theta )}{\int }_{-\infty }^{\infty }{e}^{-itx}\sum _{j=1}^4{a}_j{\phi }_{{\theta }_j}(t)dt.$$

$$\sum _{j=1}^4{a}_j{\psi }_{{\theta }_j}(t)=0,\text{for all}t,$$

$$a_2{\psi }_{\sigma }+a_4{\psi }_{\alpha }=|t{|}^{\alpha }\{-a_2\alpha +i\mathrm{sign}t(a_2\alpha \beta +a_4)\tan (\pi \alpha /2)\}.$$

$$({\psi }_{\mu },{\psi }_{\sigma },{\psi }_{\alpha },{\psi }_{\beta })=(it,-\alpha |t{|}^{\alpha },-|t{|}^{\alpha }\log |t|,i(t^{\alpha }-t)\tan (\pi \alpha /2)).$$

Full text

*Key words. * characteristic function, stable distributions, Fisher information matrix, maximum likelihood estimator, score functions.

Muneya Matsui’s research is partly supported by the JSPS Grant-in-Aid for Young Scientists B (16k16023).

1. Introduction

Stable distributions constitute a class of limit distributions of generalized central limit theorem, including the normal distribution on the border. Except for Gaussian, they do not have the second moment and the class is crowned as a representative of heavy tailed distributions. Moreover, they allow skewness and changes in supports depending on parameters. Due to such a variety of characteristics, they play crucial roles in both theory and applications. Many statistical models adopt stable random variables (r.v.’s for short) for their random components. However there is a well-known bottleneck in applications, namely most stable laws have no closed form density functions and only their characteristic functions (ch.f.’s) are explicit. Thus we need to devise methods whenever the stable laws are applied. For more details and other notable properties, consult, e.g. [18] and references therein.

In applications, several parameterizations ($A,B,C,E$ and $M$ by [21] and their variants by [15]) are available in terms of the ch.f. They have both strong and weak points in each. We leave detailed explanations to the references [21, 18, 15, 17]. Our focus here is on a continuous parameterization, which is a modified version of $(M)$ by [15]111At $\alpha=1,\,\beta\neq 0$, Zolotarev’s original $(M)$ parameterization is not a location-scale family and Nolan considered a modified version called $(0)$ parameterization. We reconcile them and use $M_{0}$ parameterization here. . We call it $(M_{0})$ parameterization. This has desirable properties in statistical applications. It is a location-scale family, and moreover, its characteristic function (ch.f.) is continuous with respect to all stable parameters $(\mu,\sigma,\alpha,\beta)$, so that we could treat the distribution continuously in the whole parameter space. Several papers recommend to use the representation for statistical applications (see, e.g. [15] and [17, Chap.I]).

Concerning the maximum likelihood estimation (MLE) for stable laws, DuMouchel has theoretically investigated the asymptotics with $(B)$ expression ([4]) and calculated the Fisher information for $(A)$ form ([5]) 222To be accurate, the definition of $(B)$ in [4] is slightly different from our version of $(B)$ by [21]. However, since we could simply imitate the theoretical approach in [4] for the theory of our version, we do not distinguish the two forms here. See Section 5 for more details. Unless parameters are in the neighborhoods of $(\alpha,\beta)=(1,0)$ or the boundaries, the asymptotics for $(B)$ are easily converted to those for $(A)$. Indeed numerical results [5] of $(A)$ are based on the theory in [4]. For this reason we could say that theoretical studies are sufficient for applications of $(A)$ and $(B)$ types.

However, as far as we know, there are no concrete asymptotic theories for MLE with $(M_{0})$ form, though possibility is suggested in [4]. Even when the $(M_{0})$ type stable law was used, only DuMouchel [4] has been referred (see, e.g. [16] or [1], we also personally communicated with John Nolan).

In this paper we analyze the asymptotics of MLE for $(M_{0})$ parameterization. More precisely, we present the consistency and asymptotic normality of MLE. Our main tools for deriving asymptotic are the detailed analysis of score functions and their derivatives. We rigorously show that the scores so that the Fisher informations are continuous at $\alpha=1$. The difficulty there is that the score functions include multiple diverging terms, which are proved to be canceled each other out. Since the case $\alpha=1,\beta\neq 0$ is excluded for $(A)$ and $(B)$ types due to discontinuity, the obtained results contrast with established asymptotics by [4].

In derivation of the asymptotics, we have to start with properties of the $(M_{0})$ density and its derivatives, since the previous investigation has been done with $(B)$ type, which has a very convenient ch.f. form for the density analysis (see [21]). We go back to the ch.f. for $(M_{0})$, from which we derive necessary properties of the density for the asymptotics of MLE. In a part of the process, we effectively use a relation of $(M_{0})$ and $(B)$ on possible parameter regions333Notice that the $(B)$ type in [4] is different from our $(B)$ form and thus even for our $(B)$ type, we need to derive necessary properties for the asymptotics separately.. Our theoretical base is a rather modern and sophisticated one [19], which is relatively easy to handle. Therefore the established theories could be developed and arranged in various ways for related applications.

Preliminary results are obtained in Section 2. For $\alpha\neq 1$ the tail behaviors of derivatives with $(M_{0})$ form are derived via those of $(B)$ form, whereas at $\alpha=1$, these quantities are independently derived. A combination of these preliminary results constitutes the tail behaviors of scores (Proposition 2.3). The consistency and asymptotic normality are presented in Section 3, which are our main results. In order to check the continuity of the Fisher information around $\alpha=1$, a small numerical work is conducted in Section 4. We discuss new and known things in the literature in Section 5, where several future works are suggested.

2. Preliminary results

This preliminary starts with characteristic function (ch.f. for short) of $(M_{0})$ parameterization and its several properties. Then we proceed to the tail behaviors of $(M_{0})$ density and its derivatives (Lemma 2.1), which are combined for analyzing the tail behaviors of score functions (Proposition 2.3).

Denote ch.f. of $(M_{0})$ parameterization by

[TABLE]

where $\mu\in{\mathbb{R}},\,\sigma\in{\mathbb{R}}_{+}\,,\alpha\in(0,2]\,,\beta\in[-1,1]$ with ${\mathbb{R}}_{+}=(0,\infty)$. We denote this parameter space by $\Theta_{M}$ and its interior by $\Theta_{M}^{\circ}$. The expression (2.1) shows continuity in $\alpha$. We see that the density $f$ is a location-scale family. Indeed the inversion formula for $\alpha\neq 1$ yields

[TABLE]

In a similar way or by continuity, we can check it also at $\alpha=1$.

We use the following notations throughout. A parameter vector is denoted by $\theta=(\theta_{1},\theta_{2},\theta_{3},\theta_{4})^{\prime}=(\mu,\sigma,\alpha,\beta)^{\prime}$. As usual $f^{\prime},f^{\prime\prime}$ mean the first and the second derivatives with respect to (w.r.t.) $x$ and $f_{\theta}=(f_{\theta_{1}},f_{\theta_{2}},f_{\theta_{3}},f_{\theta_{4}})^{\prime}$ denotes a vector of partial derivatives of $f$ w.r.t. $\theta$. The second order partial derivatives w.r.t. $x$ and $\theta$ are denoted by

[TABLE]

i.e. all derivatives will be shown to be interchangeable in our case. Moreover, $\varphi_{\theta_{i}},\,\varphi_{\theta_{i}\theta_{j}}$ respectively denote the first and the second derivatives of $\varphi$. As is well known, $\varphi_{\theta_{i}},\,\varphi_{\theta_{i}\theta_{j}}$ are represented by those of cumulant $\psi(t)=\log\varphi(t)$: $\varphi_{\theta_{i}}=\psi_{\theta_{i}}\varphi$ and $\varphi_{\theta_{i}\theta_{j}}=(\psi_{\theta_{i}\theta_{j}}+\psi_{\theta_{j}}\psi_{\theta_{j}})\,\varphi$, where $\psi_{\theta_{i}},\,\psi_{\theta_{i}\theta_{j}}$ are derivatives of $\psi$.

2.1. Behavior of derivatives of density $f$

w.r.t. $\theta$ and $x$

Here we check continuous differentiablity of $f$ and obtain tail bounds for derivatives. In the derivation of bounds, we use a relation between $(M_{0})$ and $(B)$ forms, which are possible on a restricted parameter space. We obtain tail bounds in $(B)$ form (Appendix A.2) and exploit them for finding bounds in $(M_{0})$ form. If we could not use the relation, we directly obtain bounds from derivatives of ch.f. of $(M_{0})$, which is done via the inversion formula. Recall that $\Theta_{M}$ is our parameter space and $\Theta_{M}^{\circ}$ is its interior.

Lemma 2.1**.**

For every $x\in{\mathbb{R}}$, $f(x;\theta):\theta\in\Theta_{M}^{\circ}$ is twice continuously differentiable w.r.t. $\theta$, and $f_{\theta_{i}},\,i=1,\ldots,4$ is continuously differentiable w.r.t. $x$. Moreover $f_{\theta_{i}},\,f_{\theta_{i}}^{\prime},\,f_{\theta_{i}\theta_{j}}\,i,j=1,\ldots,4$ are jointly continuous in $(x,\theta)$ on ${\mathbb{R}}\times\Theta_{M}^{\circ}$.

The tails of $f$ and its derivatives for sufficiently large $|x|$ satisfy

[TABLE]

Furthermore for $\alpha=1,\,\beta\in(-1,1)$, we have

[TABLE]

Notice that these orders are upper bounds and could possibly be smaller depending on parameter regions. For instance, in the symmetric case $\beta=0$, we could obtain better orders.

Proof.

First we assume $\alpha\neq 1$. For twice continuous differentiability of $f$, we observe derivatives of the inversion form

[TABLE]

where differentiations are done under the integral sign. Indeed, since $\varphi_{\theta_{i}\theta_{j}}$ is constructed with $e^{-|t\sigma|^{\alpha}}$ multiplied by a linear combination of powers of $|t|$ and $\log|t|$, the absolute values of integrands are integrable regardless of value of $x\in{\mathbb{R}}$ (see Lemma A.1 for exact forms of $\psi_{\theta_{i}\theta_{j}}$, so that $\varphi_{\theta_{i}\theta_{j}}$). In the form (2.5) it is not difficult to see continuity of $f_{\theta_{i}\theta_{j}}$ in $(\theta_{i},\theta_{j})$ by the dominated convergence theorem (DCT for abbreviation).

For differentiability of $f_{\theta_{j}}$ w.r.t. $x$, similarly as before, it suffices to look definability and continuity of forms

[TABLE]

where $\varphi_{\theta_{i}}$ contains the term $e^{-|t\sigma|^{\alpha}}$ and each integrand of right side is absolutely integrable. Since integrands of (2.6) are continuous in $x$, the result follows from DCT.

We take up $f_{\theta_{i}}$ and show continuity in $(x,\theta)\in{\mathbb{R}}\times\Theta_{M}^{\circ}$. With another point $(y,\theta^{\prime})\in{\mathbb{R}}\times\Theta_{M}^{\circ}$, we write

[TABLE]

where an inequality $|1-e^{it(x-y)}|\leq c|t(x-y)|^{\gamma},0<\gamma\leq 2,\,c>0$ yields a dominant function of the first part, while the second part is continuous in $\theta$ regardless of $y$. Thus again by DCT we obtain joint continuity. To make sure, we present the exact forms of $\psi_{\theta_{i}},\psi_{\theta_{i}\theta_{j}}$ so that $\varphi_{\theta_{i}},\varphi_{\theta_{i}\theta_{j}}$ in Lemma A.1. We omit the proof for $f_{\theta_{i}}^{\prime},\,f_{\theta_{i}\theta_{j}}$ which is similar.

When $\alpha=1$, we need a special treatment, since as $\alpha\to 1$ several terms in $\varphi_{\theta_{i}},\varphi_{\theta_{i}\theta_{j}}$ are diverging to $\infty$, which are proved to be canceled one another in the end. This is done in Lemma A.1, where we could see joint continuity of $\varphi_{\theta_{i}}(t),\,t\varphi_{\theta_{i}}(t)$ and $\varphi_{\theta_{i}\theta_{j}}(t)$ in $(\theta,t)\in\Theta_{M}^{\circ}\times{\mathbb{R}}$. Moreover, all of these quantities, as functions of $t$, have dominant integrable functions (Lemma A.2). Therefore, we can reuse the proof in case $\alpha\neq 1$ also for $\alpha=1$. We omit further details.

Next we proceed to the tail bounds. We start with the case $\alpha\neq 1$. Since there is the relation between $(M_{0})$ and $(B)$ forms (Lemma A.5), it suffices to use the tail bounds of $(B)$ in Lemma A.3. Namely we choose maximum tail functions among (A.9) in the expressions (A.5).

When $\alpha=1$ the proof is more complicated and we only state the outline taking up $f_{\theta_{i}}$. Proofs for other quantities $f_{\theta_{i}}^{\prime},f_{\theta_{i}\theta_{j}}$ are similar. We notice that characteristic functions of $(M_{0})$ form $\varphi$ and $(B)$ form $\varphi_{B}$ (see (2.1) and (A.5)) differ only in the constant $\pi/2$ of scale parameter, so that $\varphi$ could be analytically extended to the complex plane, which is done for $\varphi_{B}$ in [21, Ch.2]. We also apply this extension to $e^{-itx}\varphi_{\theta_{i}}(t)$ and consider the contour integration as in [21, Theorem 2.5.4]444Notice that there are several flows in the proof of Theorem 2.5.4 in [21], however, we check that the method is correct.. Then for $x>0$ and $\beta\in(-1,1)$ we have

[TABLE]

Now due to Taylor expansion of

[TABLE]

we could take the dominant term as $x\to\infty$ and obtain the tail behaviors. The exact forms of $\psi_{\theta_{i}},\,\psi_{\theta_{i}\theta_{j}}$ are given in Lemma A.1. In the case $x<0$, we use the relation

[TABLE]

where we take the complex conjugate of (2.7). Now replacing $\psi_{\theta_{i}}$ by $\overline{\psi}_{\theta_{i}}$ and $\beta$ by $-\beta$ in (2.7), we could apply the former method. ∎

Remark 2.2**.**

Results in Lemma 2.1 might be obtained directly from derivatives of the inversion form for $(M_{0})$ expression (2.2). Namely, after partially differentiate ch.f. $\varphi$ of (2.1) we could apply asymptotic expansions to the inversion formula of partial derivatives. However these expansions might require a systematic treatment of complex contour integrals as done in [21] with $(B)$ expression, which is quite long. This is a challenge for future.

2.2. Behavior of score functions and their derivatives

w.r.t. $\theta$ and $x$

Now we study score functions and their derivatives. Denote the log-likelihood function of $f$ and its scores respectively by

[TABLE]

where for convenience we sometimes write $\ell(x)$ and $\ell_{\theta_{i}}(x)$ for these quantities. The second order derivatives of score functions w.r.t. $\theta$ and $x$, denoted by

[TABLE]

are also investigated. Here orders of partial derivatives w.r.t. $(x,\theta)$ are all exchangeable. These quantities are inevitable for statistical applications other than the proof of asymptotics of MLE such as statistics where estimated parameters are inserted. We rigorously show the definability and properties of (2.8) - (2.10).

Proposition 2.3**.**

Let $\theta\in\Theta_{M}^{\circ}$. For every $x\in{\mathbb{R}}$,

[TABLE]

are well-defined and continuous in $\theta$, and they are jointly continuous in $(x,\theta)$ on ${\mathbb{R}}\times\Theta_{M}^{\circ}$. Concerning tail behaviors, we have for sufficiently large $|x|,\,x\in{\mathbb{R}}$,

[TABLE]

and moreover,

[TABLE]

Notice that results (2.12) and (2.13) are upper bounds, so that we could obtain sharper results depending on parameter values. For example, in the symmetric case $(\beta=0)$ the results are more explicit (see [7]).

Proof.

The proof for properties of (2.11) follows from Lemma 2.1 together with definitions (2.8) - (2.10). Notice that $\alpha$-stable distributions are unimodal and for every $x\in{\mathbb{R}}$, $f(x;\theta)\neq 0$ on $\theta\in\Theta_{M}^{\circ}$, and thus continuity of $f,\,f_{\theta_{i}},\,f_{\theta_{i}}^{\prime},\,f_{\theta_{i}\theta_{j}}$ in $\theta\in\Theta_{M}^{\circ}$ yields that of scores and their derivatives (2.11). Moreover, $f,\,f_{\theta_{i}},\,f_{\theta_{i}}^{\prime},\,f_{\theta_{i}\theta_{j}}$ are jointly continuous in $(x,\theta)\in{\mathbb{R}}\times\Theta_{M}^{\circ}$, so that the joint continuity of (2.11) follows.

Next we prove (2.12) and (2.13). Notice that the tail order of $f$ in Lemma 2.1 is exact, i.e. $f(x)\sim c|x|^{-1-\alpha},\,c>0$ as $|x|\to\infty.$ By substituting the result (2.3) of Lemma 2.1 into the definitions (2.8) - (2.10), we could bound them from the upper side for sufficiently large $|x|$. Thus we could easily reach (2.12) and (2.13). ∎

3. Consistency and asymptotics normality of MLE

For asymptotics of maximum likelihood estimate, we rely on a series of theorems in [19] adopted to our present situation. We are starting with additional notations. Let $P_{\theta}:\theta\in\Theta_{M}$ denote the probability measure of stable law with $(M_{0})$ parameterization. Let $X_{1},X_{2},\ldots,X_{n}$ be an iid sample from $(P_{\theta}:\theta\in\Theta_{M})$ with generic r.v. $X$. The log likelihood based on $n$ samples is given by

[TABLE]

so that its expectation is $L(\theta):={\mathbb{E}}[\ell(X;\theta)]$. A maximizer of $L_{n}(\theta)$ w.r.t. $\theta$ is denoted by $\hat{\theta}_{n}=(\hat{\mu}_{n},\hat{\sigma}_{n},\hat{\alpha}_{n},\hat{\beta}_{n})^{\prime}$. We sometimes write scores as a vector $\ell_{\theta}=(\ell_{\mu},\ell_{\sigma},\ell_{\alpha},\ell_{\beta})^{\prime}$. Since we do not know all behaviors of $\ell,\,L$ and $L_{n}$ at the boundary $\partial\Theta_{M}$, our asymptotics are formally done on arbitrary compact sets $C\subset\Theta_{M}^{\circ}$ such that the true parameter $\theta_{0}$ is included. Our main theorem is as follows.

Theorem 3.1**.**

*Let $\hat{\theta}_{n}$ be the maximum likelihood estimator based on i.i.d. $n$ observations from stable law $(P_{\theta}:\theta\in\Theta_{M})$. Assume that the true parameter $\theta_{0}$ is in the interior $\theta_{0}\in\Theta_{M}^{\circ}$ and prepare an arbitrary compact set $C\subset\Theta_{M}^{\circ}$ such that $\theta_{0}\in C$. Then MLE $\hat{\theta}_{n}$ restricted on $C$ is consistent and has asymptotic normality. In particular we have an expression *

[TABLE]

where $\sqrt{n}\,(\hat{\theta}_{n}-\theta_{0})\stackrel{{\scriptstyle d}}{{\to}}N(0,I_{\theta_{0}}^{-1})$ as $n\to\infty$, and $I_{\theta_{0}}$ is the Fisher information matrix.

One may think that preparation of a compact set $C$ is a bit strange. However a similar constraint is imposed on $\hat{\theta}_{n}$ in [4], since $L_{n}(\theta)$ in $(B)$ form possibly diverges at the boundary (see also argument (3) in Section 5). Notice that with $(M_{0})$ parameterization, we can have both consistency and asymptotic normality on the whole interior of parameter space $\Theta_{M}$. This is not possible with $(A)$ and $(B)$ expressions since they have discontinuity at $\alpha=1$.

We give the proof of consistency and that of asymptotic normality separately. For consistency we rely on [19, Therem 5.7] adopted for our purpose, which is

Theorem 3.2**.**

Suppose that for every $\varepsilon>0$

[TABLE]

where $d$ is a metric of the parameter space. Then any sequence of estimators $\hat{\theta}_{n}$ with $L_{n}(\hat{\theta}_{n})\geq L_{n}(\theta_{0})-o_{P}(1)$ converges in probability to $\theta_{0}$. Here $o_{P}(1)$ denotes a sequence of r.v.’s converging to zero in probability.

Proof of consistency.

We will check the conditions of Theorem 3.2. The inequality (3.4) is equivalent to the fact: the point $\theta_{0}\in C$ as a maximizer of continuous function $L(\theta)$ is unique. This is shown by checking the identifiability condition, i.e. $f(\cdot;\theta)\neq f(\cdot;\theta^{\prime})$ for $\theta\neq\theta^{\prime}$ (see [19, Lemma 5.35]). However, since for $\theta\neq\theta^{\prime}$ the corresponding ch.f.’s are different, the identifiability follows by the uniqueness of the Fourier transform.

An equivalent condition for (3.3) is that a set of functions $\theta\mapsto L(x;\theta),\,\theta\in C$ is Glivenko-Cantelli. This is implied by the following two conditions: functions $\theta\mapsto\log f(x;\theta)$ are continuous for every $x$ and they are dominated by an integrable envelope function (see [19, p.46] cf. [20, Ex. 3.7.3]). However these conditions are implied by Lemma 2.1 and Proposition 2.3. ∎

For the proof of asymptotic normality, we again rely on an auxiliary lemma, which is a combination of Theorem 5.39 and Lemma 7.6 in [19]. The lemma is given for a general law $(P_{\theta}:\theta\in\Theta)$ with density $p_{\theta}(x)$ and $\Theta\subset{\mathbb{R}}^{k}$ is a given parameter space.

Lemma 3.3**.**

*For the model $(P_{\theta}:\theta\in\Theta)$ with density $p_{\theta}(x)$, we assume that the map $\theta\mapsto\sqrt{p_{\theta}(x)}$ is continuously differentiable for every $x$. Suppose that the elements of the Fisher information matrix $I_{\theta}$ are well defined and continuous in $\theta$. For an inner point $\theta_{0}$ of $\Theta$, we further assume that there exists a measurable function $\dot{\eta}$ with ${\mathbb{E}}_{\theta_{0}}[\dot{\eta}^{2}]<\infty$ such that for every $\theta^{\prime}$ and $\theta^{\prime\prime}$ in a neighborhood of $\theta_{0}$, *

[TABLE]

where $\|\cdot\|$ is the Euclidean norm. If $I_{\theta_{0}}$ is nonsingular and $\hat{\theta}_{n}$ is consistent for $\theta_{0}$, then

[TABLE]

In particular, the sequence $\sqrt{n}\,(\hat{\theta}_{n}-\theta_{0})$ is asymptotically normal with mean zero and covariance matrix $I_{\theta_{0}}^{-1}$.

Notice that first two conditions of Lemma 3.3 are sufficient for the “differentiable in quadratic mean” condition in [19, Theorem 5.39], which is the main assertion of [19, Lemma 7.6].

Proof of asymptotic normality.

We check the conditions of Lemma 3.3 step by step. Lemma 2.1 implies continuous differentiability of $\sqrt{f(x;\theta)}$ for every $x$. We see elements of the Fisher information matrix

[TABLE]

In view of Proposition 2.3, $\ell_{\theta}(x)$ are continuous in $\theta$ for every $x$ and, moreover, by tail conditions of $\ell_{\theta_{i}}$ and $f$, all $4\times 4$ integrands have dominating functions which are absolutely integrable. Thus continuity of $I_{\theta}$ in $\theta$ follows from the dominating convergence theorem. In order to check (3.5), we apply the mean value theorem to obtain

[TABLE]

where $\theta_{i}^{\ast}$ are vectors between $\theta^{\prime}$ and $\theta^{\prime\prime}$. In the left side, we have ${\mathbb{E}}[(\sum_{i=1}^{4}\ell_{\theta_{i}}(X;\theta_{i}^{\ast}))^{2}]<\infty$ since for any $\theta_{i}^{\ast},\theta_{j}^{\ast}\in C$, ${\mathbb{E}}[|\ell_{\theta_{i}}(X;\theta_{i}^{\ast})\ell_{\theta_{j}}(X;\theta_{j}^{\ast})|]<\infty,\,i,j=1,\ldots,4$ follow from tail behaviors of $\ell_{\theta}$ in Proposition 2.3. Thus (3.5) follows.

We proceed to the nonsingularity of $I_{\theta_{0}}$. We take a similar approach as in [4] and prepare linear combinations of scores ${\bf a}^{\prime}\ell_{\theta}=\sum_{j=1}^{4}a_{j}\ell_{\theta_{j}}(x)$ where ${\bf a}=(a_{1},a_{2},a_{3},a_{4})^{\prime}\in{\mathbb{R}}^{4}$. Since ${\mathbb{E}}[({\bf a}^{\prime}\ell_{\theta}(X))^{2}]$ constitutes a quadratic form of $I_{\theta}$, it suffices to show that $(\ell_{\theta_{i}})$ are linearly independent, namely ${\bf a}^{\prime}\ell_{\theta}(x)=0$ for all $x$ if and only if ${\bf a}$ is zero vector. In what follows we assume the former and derive the latter since the opposite direction is obvious. We use the inversion formula and write

[TABLE]

By the uniqueness of the Fourier transform, the assumption ${\bf a}^{\prime}\ell_{\theta}(x)=0$ for all $x$ is equivalent to that $\sum_{j=1}^{4}a_{j}\varphi_{\theta_{j}}(t)=0$ for all $t$. This implies

[TABLE]

from which we show that ${\bf a}$ is zero vector. We analyze (3.9) using expressions of $\psi_{\theta_{k}}$ in Lemma A.1. We start with the case $\alpha=1,\,\beta\neq 0$. Only $\psi_{\alpha}$ has the term $|t|^{\alpha}\log|t|$ and others do not, so that $a_{3}=0$. We focus on $a_{2}\psi_{\sigma}+a_{4}\psi_{\alpha}$ and collect terms related with $|t|^{\alpha}$, which is

[TABLE]

Then we should have $a_{2}=a_{4}=0$, since both real and imaginary parts need to be zero. When $\alpha\neq 1,\,\beta=0$ we have a simpler form

[TABLE]

Similarly as before $a_{3}$ should be zero. Since we could not cancel $|t|^{\alpha}$ of $\psi_{\sigma}$ by a linear combination of $\psi_{\mu}$ and $\psi_{\beta}$, it should be $a_{1}=a_{2}=0$.

Next we consider the case $\alpha=1,\,\beta\neq 0$ with expressions in (A.1). Sine $\psi_{\alpha}$ includes $\log^{2}|t|$ and others do not, $a_{3}$ should be zero. We focus on $t\log|t|$ in $\psi_{\sigma}$ and $\psi_{\beta}$ and find $a_{4}=-\beta a_{2}$. Then from $\psi_{\mu}$ and $\psi_{\sigma}$, we have $ita_{1}=(|t|+i(2\beta/\pi)t)a_{2}$, which is not possible unless $a_{1}=a_{2}=0$. In the case $\alpha=1,\,\beta=0$, we have

[TABLE]

If we look the pairs $(\psi_{\mu},\psi_{\sigma})$ and $(\psi_{\alpha},\psi_{\beta})$, one element of each pair includes the absolute value $|t|$ and the other does not. Thus they are linearly independent. ∎

Notice that the proof at $\beta=0$ is not implied by asymptotics of symmetric case since $\beta$ is not estimated there.

4. Fisher information around the Cauchy law

For confirmation, we numerically examine smoothness of $\varphi_{\theta_{i}}$ and $f_{\theta_{i}}$ at $\alpha=1$. By using these quantities, we approximate the Fisher information matrix $I_{\theta}$ around the Cauchy law $(\alpha,\beta)=(1,0)$. We could not see any discontinuous behaviors of elements in $I_{\theta}$ as $\alpha\to 1,\,\beta\neq 0$ around the Cauchy law, $\beta\in(-\varepsilon,\varepsilon),\,\varepsilon>0$ , and we observe continuity of all the elements of $I_{\theta}$ at $(\alpha,\beta)=(1,0)$.

For convenience the standard $(\mu,\sigma)=(0,1)$ case is considered and we sometimes write $f(x;\alpha,\beta)$ omitting the location and scale. Since $f$ is known to be continuous in $(\alpha,\beta)$ on $\Theta_{M}^{\circ}$, in $I_{\theta_{i}\theta_{j}}$ of (3.6) we use the Cauchy law for $f$, whereas for $f_{\theta}$ we take the exact one with the inversion expression, namely for the integrands we consider

[TABLE]

For integration of (4.1), we apply the derivatives of the Dirac delta $\delta$: $\delta^{(n)}(y)=i^{n}/(2\pi)\int_{-\infty}^{\infty}x^{n}e^{ixy}dx$ for $n\in{\mathbb{N}}$ and their property: for $n$ times continuously differentiable $h$, $\int_{-\infty}^{\infty}\delta^{(n)}(t)h(t)dt=(-1)^{n}h^{(n)}(0)$. Now applying

[TABLE]

we obtain through Fubini’s theorem and change of variables that

[TABLE]

where the exact forms for $\varphi_{\theta_{i}}$ and $\varphi_{\theta_{i}}^{\prime}$ are recovered by Lemma A.1. We also evaluate errors of our approximation. Taylor’s expression around the Cauchy density yields

[TABLE]

where $(\alpha^{\ast},\beta^{\ast})$ is a value between $(1,0)$ and $(\alpha,\beta)$, which also may depend on $x$. Then approximation errors for (4.1) are

[TABLE]

The integral of the left sides are the errors $I_{\theta_{i}\theta_{j}}-\tilde{I}_{\theta_{i}\theta_{j}}$. In view of Lemma 2.1 and Proposition 2.3, the integrands in error terms are uniformly integrable in $(\alpha,\beta)$ close to $(1,0)$. Thus the order of errors is $O(|\alpha-1|)+O(|\beta|)$.

In Table 1 we present the exact value of $I_{\theta}$ at $(\alpha,\beta)=(1,0)$ from (4.2).

Our values are consistent with those in [5], [7] and the values given personally by John Nolan which are obtained with improvements in the method of [14]. Indeed we could obtain exact values. Let $\gamma\doteq 0.57722$ be Euler’s constant and we have $I_{\mu\sigma}=I_{\mu\alpha}=I_{\alpha\beta}=I_{\sigma\beta}=0$, $I_{\mu\mu}=I_{\sigma\sigma}=0.5$,

[TABLE]

Concerning the approximation of $I_{\theta}$, our numerical study (4.2) precisely reflects the theory. Namely we numerically confirmed that the Fisher information $I_{\theta}$ is continuous at $\alpha=1,\,\beta\neq 0$, although the study is around the Cauchy law. Moreover, $I_{\theta}$ is continuous at $(\alpha,\beta)=(0,1)$ as a function of all four parameters. We illustrate the elements $I_{\alpha\alpha}$ and $I_{\beta\beta}$ for the range $0.95\leq\alpha\leq 1.05$ and $-0.1\leq\beta\leq 0.1$ in Table 2, where values are symmetric about $\beta=0$ and we omit the case $\beta\leq 0$. Even when $\alpha=1$ we do not observe large values, which contrasts with the Fisher information matrix for $(A)$ form by [4].

5. Discussion and future works

Before we describe preceding and future works we make a remark on the parameterization $(B)$. Recall that the form of $(B)$ in [4] and that of ours are slightly different. To be more precise, only the skewness parameter $\beta_{B}^{\prime}$ in [4] is different and the others are the same. They are connected by

[TABLE]

so that the parameter range of $\beta_{B}^{\prime}$ is $|\beta_{B}^{\prime}|\leq|K(\alpha)|$. In view of the above relation, the asymptotics of MLE in [4] are slightly simpler than our version of $(B)$ since the parameter $\alpha$ appears just once in ch.f. of [4] (cf. (A.5)). Although in practice MLE of $\hat{\beta}^{\prime}_{B}$ may be more affected by $\hat{\alpha}$ close to the boundary, the asymptotic theories of both forms are almost the same. Thus we do not distinguish two versions in our paper.

Next we present past researches and discuss about future works. In this paper, we analyzed the sores and related functions for $(M_{0})$ form on the interior of the parameter space. Our particular interest is on their tail behaviors, from which we have derived asymptotics of MLE. As stated in the introduction, these investigations are sufficient for $(A)$ and $(B)$ types unless parameters are in the neighborhoods of $\alpha=1$ or the boundaries.

Then next natural questions are what are the behaviors of densities and scores around the boundaries. These are crucial in statistical applications. In what follows, we clarify related preceding researches as possible as we could, focusing on $(A),(B)$ and $(M_{0})$ parameterizations. Notice that this is not a complete list and we possibly overlook some references555 Notice that we confine the list to theories for MLE and calculations of the related Fisher information matrix. We omit the literature of statistical estimation methods since they are too many.. We are welcome for any comments. For convenience, the parameters are denoted by $(\mu_{i},\sigma_{i},\alpha,\beta_{i}),\,i=A,B,M_{0}$ and we take the standard cases $(\mu_{i},\sigma_{i})=(0,1)$. Note that three parameterizations are the same in the symmetric case $\beta_{i}=0$.

1. (1)

Case $\alpha$ close to $2$. In [4, p.955] it is pointed out that the Fisher information for $\alpha$ (denoted by $I_{\alpha\alpha}$) diverges to $\infty$ as $\alpha\to 2$ in $(B)$ forms. This fact has also been numerically examined with $(A)$ form in [5, Table 1,2A] and with $(M_{0})$ form in [16, Sec.4]. These observations are theoretically supported. Indeed, the rate of divergence of $I_{\alpha\alpha}$ has been derived in the symmetric case ([11]). Succeeding the idea of [11], the diverging speed in the non- symmetric case has been studied in $(M_{0})$ form (see [6]). For the symmetric case, the Fisher information matrix and MLE around $\alpha=2$ have been numerically studied in quite some detail (see [7]). 2. (2)

Case $\alpha$ close to $1$ and $\beta\neq 0$. Although the discontinuity at $\alpha=1,\beta\neq 0$ is known for parameterizations $(A)$ and $(B)$, the behaviors of scores around $\alpha=1$ have not been analytically investigated. Notice that the limit distributions at $\alpha=1$ exhibit quite different patterns depending on the parameterization (see [21, p.11,12]). In [5] the information matrix around $\alpha=1$ has numerically been studied, where $I_{\alpha\alpha},\,I_{\beta_{A}\beta_{A}}$ and $I_{\sigma_{A}\sigma_{A}}$ showed quite large values. 3. (3)

Case $\alpha$ close to [math]. According to [4, p.955] in $(B)$ form, the likelihood function w.r.t. $\alpha$ and $\mu_{B}$ has no maximum on $\alpha\in(0,2]$ and $\mu_{B}\in{\mathbb{R}}$. Instead it diverges to $\infty$ as $(\alpha,\mu_{B})\to(0,x_{k})$ where $x_{k}$ is an observed sample. In the symmetric case, the divergence of $I_{\alpha\alpha}$ as $\alpha\to 0$ has theoretically been proved (see Theorem 2 in [13]). 4. (4)

Case $\beta$ close to $\pm 1$. It is also pointed out in [4, p.955] that as $\beta_{B}\to\pm 1$, the Fishier information $I_{\beta_{B}\beta_{B}}$ approaches $\infty$. For $\alpha\in(1,2)$, the tail behaviors as $\beta_{B}\to\pm 1$ are derived in [12].

Now one finds that regardless of the parameterization not all boundary cases have been analytically studied. Here the boundary cases imply that $\alpha=0,1,2,\,\beta_{i}=0,\pm 1,\,i=A,B,M_{0}$ and their combinations. Since stable r.v.’s are assumed in random quantities of many statistical models, in view of its importance, further investigations are required in both theory and numerical works.

Finally, we mention an application in goodness-of-fit tests for stable laws. Usually these kinds of tests are done with empirical ch.f.’s since most stable laws have only closed form density expressions. Then asymptotics of empirical ch.f.’s are needed (see [9, 8, 10]). When parameters are estimated, the weak convergence of empirical ch.f. is assured by conditions (vi-iv) in [3]. As a by-product of our study it is shown that those conditions are satisfied in $(M_{0})$ form. Therefore, the theme would be one of our future works.

Appendix A Technical lemmas

A.1. Derivatives of ch.f. and cumulant w.r.t. parameters

The first and the second derivatives of ch.f. $\varphi(t)$ w.r.t. $\theta$ are given by those of the corresponding cumulant $\psi(t)=\log\varphi(t)$, namely $\varphi_{\theta_{i}}=\psi_{\theta_{i}}\varphi$ and $\varphi_{\theta_{i}\theta_{j}}=(\psi_{\theta_{j}}\psi_{\theta_{j}}+\psi_{\theta_{i}\theta_{j}})\varphi$ where $\theta=(\theta_{1},\theta_{2},\theta_{3},\theta_{4})^{\prime}=(\mu,\sigma,\alpha,\beta)^{\prime}$. In what follows, we present only derivatives for $\psi$ which are results of straightforward calculations. In view of expectations in Lemma A.1 below one could observe that $\psi_{\theta_{i}}(t)$ and $\psi_{\theta_{i}\theta_{j}}(t)$ are jointly continuous in $(t,\theta)\in{\mathbb{R}}\times\Theta_{M}^{\circ}$, so are $\varphi_{\theta_{i}}$ and $\varphi_{\theta_{i}\theta_{j}}$.

Lemma A.1**.**

The first and the second derivatives of $\psi(t)=\log\varphi(t)$ are

[TABLE]

For $\alpha=1$, the derivatives of $\psi(t)$ are as follows. The quantities $\psi_{\mu},\psi_{\mu\mu},\psi_{\mu\sigma},\psi_{\mu\alpha},\psi_{\mu\beta}$ and $\psi_{\beta\beta}$ do not change, and

[TABLE]

The calculations for $\alpha=1$ are really complicated. However, one could see the basic idea is in the proof of Lemma A.2, and we omit the details.

Lemma A.2**.**

The quantities $\varphi_{\theta_{i}}(t),\,t\varphi_{\theta_{i}}(t)$ and $\varphi_{\theta_{i}\theta_{j}}(t),\,i,j=1,\ldots,4$ around $\alpha=1$ are respectively bounded by the dominant integrable functions.

Proof.

We only take up $\varphi_{\alpha}(t)$ since the proofs for other quantities are similar, though some are more complicated. Recall that $\varphi$ includes $e^{-|t|^{\alpha}}$ and $\varphi_{\alpha}=\psi_{\alpha}\varphi$. In view of $\psi_{\alpha},\,\alpha\neq 1$ in Lemma A.1, a dominant function for the term $-|t|^{\alpha}\log|t|$ is easy, whereas remaining terms include $\tan(\pi\alpha/2)$ or $\cos^{-1}(\pi\alpha/2)$ which diverges to $\pm\infty$ as $\alpha\to 1$. We focus on the remainder and write

[TABLE]

Observe the following Taylor expansions around $\alpha=1$ with error terms:

[TABLE]

where $\alpha_{i}^{\ast},\,i=1,2,3$ are values between $\alpha$ and $1$, and $\alpha_{i}^{\ast},\,i=1,2$ depend also on $t$. Applying (A.2) to $A_{\alpha}(t)$, we have for $\alpha\neq 1$

[TABLE]

where

[TABLE]

and $c_{i},\,i=1,\ldots,4$ are constants independent of $t$. Now noticing $\tan(\pi\alpha/2)\sim(2/\pi)/(1-\alpha)+O(|\alpha-1|)$, we obtain

[TABLE]

so that we reach $\psi_{\alpha}$ in (A.1). Looking $A_{\alpha}(t),\,\alpha\neq 1$ above, we observe that $\varphi_{\alpha}(t)$ is constructed with a linear combination of products by $t,\,\log^{i}|t|,\,i=1,2,3,\,|t|^{\alpha_{1}^{\ast}-1}$, $|t|^{\alpha_{2}^{\ast}-1}$ and $\varphi(t)$. Since $\varphi$ include $e^{-|t|^{\alpha}}$, we could have an integrable dominant function. ∎

A.2. Tails of density and derivatives in $(B)$ form

In this subsection we briefly explain the tail properties in $(B)$ expression, which are exploited in the main part. As stated in the introduction the form of ch.f. in $(B)$ type is more convenient than that of $(M_{0})$ in analytic point of view. In [21] thorough ch.f. various properties of $(B)$ density have been investigated. Following [21], we treat the standard density $g(x):=g(x;\alpha,\beta_{B})$ of $(B)$ expression where the word ’standard’ implies that location and scale parameters satisfy $(\mu_{B},\sigma_{B})=(0,1)$. Here the exponent $\alpha$ is uniform $\alpha_{M}=\alpha_{\beta}=\alpha$ and $\beta_{B}$ denote the skewness parameter. We are starting with definition of ch.f.,

[TABLE]

where $K(\alpha)=\alpha-1+{\rm sign}(1-\alpha)$ (see [21, (2.2.1a),(2.2.1b)]). Then by the inversion formula

[TABLE]

we reach to the expression [21, (2.2.1), p.66], i.e.

[TABLE]

which is used in the following analysis. In (A.8) we could see the discontinuity at $\alpha=1$ and thus we restrict parameter space to $\Theta_{B}=\{\alpha,\beta_{B}\mid\alpha\in(0,1)\cup(1,2),\,\beta_{B}\in(-1,1)\}$. As for notations of derivatives, we reuse those of $f$, and write $g^{\prime},g^{\prime\prime},g_{\theta_{i}},g_{\theta_{i}}^{\prime},g_{\theta_{i}\theta_{j}},\varphi_{B,\theta_{i}}$ and $\varphi_{B,\theta_{i}\theta_{j}}$ for $i,j=3,4$, where they mean the same quantities for those of $f$ except that $g$ and $\varphi_{B}$ are always differentiated by $\beta_{B}$ (not $\beta$). We omit the capital $B$ from $\beta_{B}$ for notational simplicity.

Now we proceed to the main results.

Lemma A.3**.**

Let $(\alpha,\beta_{B})\in\Theta_{B}$. For every $x\in{\mathbb{R}}$, $g(x;\alpha,\beta_{B})$ is twice continuity differentiable w.r.t. $(\alpha,\beta_{B})$ and $x$. Moreover, $g_{\theta_{i}},\,g_{\theta_{i}}^{\prime},\,g^{\prime},\,g^{\prime\prime}$ and $g_{\theta_{i}\theta_{j}},\,i,j=1,2$ are jointly continuous on ${\mathbb{R}}\times\Theta_{B}$. The tails of the density $g$ and its derivatives satisfy for sufficiently large $|x|$,

[TABLE]

Proof.

For the first, second and joint differentiabilities, the proof is done exactly the same way as that in Lemma 2.1 and we omit it. We show the tail bounds separately for $\alpha<1$ and $\alpha>1$.

**Case $\alpha<1$. ** First we consider the case $x>0$. We use another representation of the density in $(B)$ form

[TABLE]

which is obtained by considering a contour integral on the complex plane (see [21, (2.2.8) in Theorem 2.2.1]). We only explain those of $g^{\prime\prime}$ and $g_{\alpha}$ since the other cases are similar. A straightforward calculation yields

[TABLE]

where we use Fubini’s theorem for exchange of the improper integral and the infinite sum, which is possible for sufficiently large $x$. Since we need to take the imaginary part, the integral of term $k=1$ is dominant and we get the result. Moreover,

[TABLE]

as $x\to\infty$ by DCT. The case $x<0$ is derived from those for $x>0$ by the relation in [21, p.65],

[TABLE]

Case $\alpha>1$. For $x>0$ we use the relation [21, p.94, (2.5.5)],

[TABLE]

where $\alpha^{\prime}=1/\alpha$ and $\beta_{B}^{\prime}=1-(2-\alpha)(1+\beta_{B})$. We differentiate both sides and represent partial derivatives for $\alpha>1$ by combinations of derivatives of $x^{-1-\alpha}$ and and $g(x^{-\alpha};\alpha^{\prime},\beta_{B}^{\prime})$ with $\alpha^{\prime}<1$. The partial derivatives of $g$ around the origin for $\alpha<1$ converge to constants (could be zero) as $x\to 0$, which are observed by the direct differentiation of (A.8) under the integral sign. Therefore by letting $x\to\infty$ (so that $x^{-\alpha}\to 0$), in the right side of (A.11), the tail bounds for $x>0$ are derived. These bounds depend on $\alpha$ in the same manner as in case $\alpha<1$. The case $x<0$ follows again by (A.10). ∎

Notice that bounds of (A.9) are not always exact and we could obtain better ones depending on parameters.

Remark A.4**.**

For the proof of Lemma A.3 we could alternatively exploit series expansions (2.4.6), (2.4.8), (2.5.1) and (2.5.4) in [21], some of which are obtained by [2]. In fact, the approach in [4] depends on series expansions by [2]. We have checked that term-wise differentiations are possible for these expansions and could obtain the same results. The derivation of the expansions requires a systematic treatment of the complex contour integral (see Sec.2.4,2.5 in [21]) which is a considerable burden for readers. We derive the results directly from the expressions of the inversion formula.

A.3. Expressions for derivatives of $f$ with those of $g$

Next we see the relation of $f$ and $g$. Again we take the standard density for $g$. The parameter $\alpha$ is uniform and the skewness parameters $\beta$ and $\beta_{B}$ are linked by

[TABLE]

so that $\beta\neq\pm 1\Leftrightarrow\beta_{B}\neq\pm 1$. First we express $f$ with $g$ comparing (2.2) and (A.8). In (2.2) we change variables $t\to\gamma_{\alpha,\beta}s$ where

[TABLE]

and obtain

[TABLE]

where

[TABLE]

Since $g$ is infinitely differentiable with $x,\alpha,\beta_{B}$ on ${\mathbb{R}}\times\Theta_{B}$, we can express derivatives of $f$ by those of $g$.

In what follows $f$ and its derivatives are evaluated at $(x,\theta)$ and those of $g$ are evaluated at $(x^{\ast},\alpha,\beta_{B})$ where $\beta_{B}$ is given by (A.12) and (A.13).

Lemma A.5**.**

Let $\theta\in\Theta_{M}^{\circ}\cap\{\alpha\neq 1\}$ so that $(\alpha,\beta_{B})\in\Theta_{B}$. The expression for derivatives of $f$ by those of $g$ are

[TABLE]

Acknowledgment: I would like to thank Prof. John Nolan for teaching related references and for showing numerical values of the Fisher information matrix at the Cauchy distribution.