Inequalities related to some types of entropies and divergences

TL;DR

This paper explores mathematical properties and bounds of various extended entropies and divergences, including Tsallis, biparametrical, and quantum entropies, using inequalities like Hermite-Hadamard.

Contribution

It introduces new bounds and inequalities for extended entropies and divergences, including biparametrical and quantum types, expanding theoretical understanding.

Findings

New bounds for Tsallis quasilinear entropy and divergence.

Bounds for biparametrical extended entropies and divergences.

Inequalities for extended Lin's divergence and characterizations of quantum entropies.

Abstract

The aim of this paper is to discuss new results concerning some kinds of parametric extended entropies and divergences. As a result of our studies for mathematical properties on entropy and divergence, we give new bounds for the Tsallis quasilinear entropy and divergence by applying the Hermite-Hadamard inequality. We also give bounds for biparametrical extended entropies and divergences which have been given in \cite{7}. In addition, we study $(r,q)$-quasilinear entropies and divergences as alternative biparametrical extended entropy and divergence, and then we give bounds for them. Finally we obtain inequalities for an extended Lin's divergence and some characterizations of Fermi-Dirac entropy and Bose-Einstein entropy.

Equations

$${\ln }_{q}x={\int }_0^1{x}^{(1-q)t}\log xdt(x>0,q\ne 1)$$

$$M_{\psi }(x_1,x_2,...,x_n)={\psi }^{-1}\left(\sum _{j=1}^n{p}_j\psi (x_j)\right),$$

$$H_q(p_1,\cdots ,p_n)=\sum _{j=1}^n\frac{p_j-p_j^q}{1-q}=-\sum _{j=1}^n{p}_j^q{\ln }_q{p}_j=\sum _{j=1}^n{p}_j{\ln }_q\frac{1}{p_j},$$

$$\underset{q\to 1}{\lim }{H}_q(p_1,\cdots ,p_n)=H_1(p_1,\cdots ,p_n)=H(\mathbf{p}):=-\sum _{j=1}^n{p}_j\log p_j.$$

$$I_{1}^{\phi}\big{(}\mathbf{p}\big{)}=-\log\phi^{-1}\bigg{(}\sum_{j=1}^{n}p_{j}\phi\big{(}p_{j}\big{)}\bigg{)}.$$

$$I_1^{\psi }(\mathbf{p})=\log {\psi }^{-1}\left(\sum _{j=1}^n{p}_j\psi \left(\frac{1}{p_j}\right)\right),$$

$$R_q(\mathbf{p})=\frac{1}{1-q}\log \left(\sum _{j=1}^n{p}_j^q\right).$$

$$D_1^{\psi }(\mathbf{p}||\mathbf{r})=-\log {\psi }^{-1}\left(\sum _{j=1}^n{p}_j\psi \left(\frac{r_j}{p_j}\right)\right).$$

$$D_{1}^{\log}\big{(}\mathbf{p}||\mathbf{r}\big{)}=-\sum_{j=1}^{n}p_{j}\log\frac{r_{j}}{p_{j}}:=D_{1}(\bf{p}||\bf{r}).$$

$$D_q^R(\mathbf{p}||\mathbf{r})=\frac{1}{q-1}\log \left(\sum _{j=1}^n{p}_j^q{r}_j^{1-q}\right).$$

$$D_1^{x^{1-q}}(\mathbf{p}||\mathbf{r})=-\log {\left(\sum _{j=1}^n{p}_j{\left(\frac{r_j}{p_j}\right)}^{1-q}\right)}^{\frac{1}{1-q}}=D_q^R(\mathbf{p}||\mathbf{r}).$$

$$D_q^T(\mathbf{p}||\mathbf{r}):=\sum _{j=1}^n\frac{p_j-p_j^q{r}_j^{1-q}}{1-q},$$

$$D_q^T(\mathbf{p}||\mathbf{r})=\sum _{j=1}^n{p}_j^q({\ln }_q{p}_j-{\ln }_q{r}_j)=-\sum _{j=1}^n{p}_j{\ln }_q\frac{r_j}{p_j}.$$

$$\underset{q\to 1}{\lim }{D}_q^T(\mathbf{p}||\mathbf{r})=D_1(\mathbf{p}||\mathbf{r})=-\sum _{j=1}^n{p}_j\log \frac{r_j}{p_j}.$$

$$D^{(\alpha )}(\mathbf{p}||\mathbf{r}):=\frac{4}{1-{\alpha }^2}\left(1-\sum _{j=1}^n{p}_j^{\frac{1-\alpha }{2}}{r}_j^{\frac{1+\alpha }{2}}\right)=D^{(\alpha )}(\mathbf{p}||\mathbf{r}),$$

$$D^{(\alpha )}(\mathbf{p}||\mathbf{r})=\frac{1}{q}{D}_q^T(\mathbf{p}||\mathbf{r})$$

$$D_q^R(\mathbf{p}||\mathbf{r})=\frac{1}{q-1}\log \left(1+(q-1)D_q^T(\mathbf{p}||\mathbf{r})\right),$$

$$D_q^R(\mathbf{p}||\mathbf{r})\ge D_q^T(\mathbf{p}||\mathbf{r}),$$

$$D_q^R(\mathbf{p}||\mathbf{r})\le D_q^T(\mathbf{p}||\mathbf{r}).$$

$$I_q^{\psi }(\mathbf{p})={\ln }_q{\psi }^{-1}\left(\sum _{j=1}^n{p}_j\psi \left(\frac{1}{p_j}\right)\right),$$

$$\underset{q\to 1}{\lim }{I}_q^{\psi }(\mathbf{p})=I_1^{\psi }(\mathbf{p}).$$

$${\exp_{q}}\left(x\right)=\left\{\begin{array}[]{l}{\left\{{1+\left({1-q}\right)x}\right\}^{\frac{1}{{1-q}}}},\,\,\,\,\,\,\,\,\,\,\mbox{if}\,\,\,1+\left({1-q}\right)x>0\\ \mbox{undefined},\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\mbox{otherwise}.\end{array}\right.$$

$$I_q^{\psi }(\mathbf{p})\ge 0.$$

$$\exp R_q(\mathbf{p})={\exp }_q{H}_q(\mathbf{p}).$$

$$1+\big{(}1-q\big{)}H_{q}(\mathbf{p})=\sum_{j=1}^{n}p_{j}^{q}>0.$$

$$D_q^{\psi }(\mathbf{p}||\mathbf{r}):=-{\ln }_q{\psi }^{-1}\left(\sum _{j=1}^n{p}_j\psi \left(\frac{r_j}{p_j}\right)\right).$$

$$\underset{q\to 1}{\lim }{D}_q^{\psi }(\mathbf{p}||\mathbf{r})=D_1^{\psi }(\mathbf{p}||\mathbf{r}):=-\log {\psi }^{-1}\left(\sum _{j=1}^n{p}_j\psi \left(\frac{r_j}{p_j}\right)\right).$$

$$D_q^{{\ln }_q}(\mathbf{p}||\mathbf{r})=-\sum _{j=1}^n{p}_j{\ln }_q\left(\frac{r_j}{p_j}\right)=D_q^T(\mathbf{p}||\mathbf{r}).$$

$$D_q^{x^{1-q}}(\mathbf{p}||\mathbf{r})=D_q^T(\mathbf{p}||\mathbf{r}).$$

$$D_q^{\psi }(\mathbf{p}||\mathbf{r})\ge 0.$$

Full text

Inequalities related to some types of entropies and divergences

Shigeru Furuichi1111E-mail:furuichi@chs.nihon-u.ac.jp and Nicuşor Minculete2222E-mail:minculeten@yahoo.com

1Department of Information Science,

College of Humanities and Sciences, Nihon University,

3-25-40, Sakurajyousui, Setagaya-ku, Tokyo, 156-8550, Japan

2Transilvania University of Braşov, Braşov, 500091, Romania

Abstract. The aim of this paper is to discuss new results concerning some kinds of parametric extended entropies and divergences. As a result of our studies for mathematical properties on entropy and divergence, we give new bounds for the Tsallis quasilinear entropy and divergence by applying the Hermite-Hadamard inequality. We also give bounds for biparametrical extended entropies and divergences which have been given in [1]. In addition, we study $(r,q)$-quasilinear entropies and divergences as alternative biparametrical extended entropy and divergence, and then we give bounds for them. Finally we obtain inequalities for an extended Lin’s divergence and some characterizations of Fermi-Dirac entropy and Bose-Einstein entropy.

**Keywords : ** Shannon entropy, divergence (relative entropy), Tsallis entropy, Rényi entropy, biparametrical extended entropy and biparametrical extended divergence

**2010 Mathematics Subject Classification : ** Primary 46C05, secondary 26D15, 26D10.

1 Introduction

Generalized entropies have been studied by many researchers (we refer the interested readers to [2]). Rényi [3] and Tsallis [4] entropies are well known as one-parameter generalizations of Shannon entropy, being intensively studied not only in the field of classical statistical physics [5, 6, 7], but also in the field of quantum physics in relation to the entanglement [8].

The Tsallis entropy is a natural one-parameter extended form of the Shannon entropy, hence it can be applied to known models which describe systems of great interest in atomic physics [9]. However, to our best knowledge, the physical relevance of a parameter of the Tsallis entropy was highly debated and it has not been completely clarified yet, the parameter being considered as a measure of the non-extensivity of the system under consideration.

One of the authors of the present paper studied the Tsallis entropy and the Tsallis divergence from a mathematical point of view. Firstly, fundamental properties of the Tsallis divergence were discussed in [10]. The uniqueness theorem for the Tsallis entropy and Tsallis divergence was studied in [11]. Following this result, an axiomatic characterization of a biparametrical extended divergence was given in [1]. In [12], information theoretical properties of the Tsallis entropy and some inequalities for conditional and joint Tsallis entropies were derived. In [13], matrix trace inequalities for the Tsallis entropy were studied. And, in [14], the maximum entropy principle for the Tsallis entropy and the minimization of the Fisher information in Tsallis statistics were studied.

Quite recently, we provided mathematical inequalities for some divergences in [15], considering that it is important to study the mathematical inequalities for the development of new entropies. We show several results from our paper [16], here we define a further generalized entropy based on Tsallis and Rényi entropies and study mathematical properties by the use of scalar inequalities to develop the theory of entropies. While we applied the Young inequality in [15] and Jensen type inequality in [16] to obtain the inequalities for entropies and divergences, we apply the Hermite-Hadamard inequality with the integral relation

[TABLE]

to obtain some new results in the present paper, where the $q$-logarithmic function is defined by $\ln_{q}(x)=\frac{x^{1-q}-1}{1-q}$ ($x>0,\,\,1\neq q>0$). We also study two different kinds of biparametical extended entropies and divergences in Section 3.

We start from the weighted quasilinear mean (see [17, p.677] for example) for some continuous and strictly monotonic function $\psi:I\to\mathbb{R}$, defined by

[TABLE]

where $\sum_{j=1}^{n}p_{j}=1,p_{j}>0,x_{j}\in I,$ for $j=1,\cdots,n$. If we take $\psi(x)=x$, then $M_{\psi}(x_{1},x_{2},...,x_{n})$ coincides with the weighted arithmetic mean $A(x_{1},x_{2},...,x_{n})=\sum_{j=1}^{n}p_{j}x_{j}$. If we take $\psi(x)=\log x$, then $M_{\psi}(x_{1},x_{2},...,x_{n})$ coincides with the weighted geometric mean $G(x_{1},x_{2},...,x_{n})=\prod_{j=1}^{n}x_{j}^{p_{j}}$. If $\psi(x)=x$ and $x_{j}=\ln_{q}{\frac{1}{p_{j}}}$, then $M_{\psi}(x_{1},x_{2},...,x_{n})$ is equal to Tsallis entropy [4]:

[TABLE]

where $\{p_{1},\cdots,p_{n}\}$ is a probability distribution with $p_{j}>0$ for all $j=1,\cdots,n$. Since the $q$-logarithmic function$\ln_{q}(x)$ uniformly converges to the usual logarithmic function $\log x$ in the limit $q\to 1$, Tsallis entropy $H_{q}(p_{1},\cdots,p_{n})$ converges to Shannon entropy $H_{1}(p_{1},\cdots,p_{n})$ in the limit $q\to 1$:

[TABLE]

Thus, it is known that Tsallis entropy is one of the generalizations of Shannon entropy. It is also known that Rényi entropy [3] is a generalization of Shannon entropy. Hereafter we use the notations $\mathbf{p}=\left\{p_{1},\cdots,p_{n}\right\}$ and $\mathbf{r}=\left\{r_{1},\cdots,r_{n}\right\}$ with $p_{j}>0$ and $r_{j}>0$ for $j=1,\cdots,n$, as probability distributions. Here, we review a quasilinear entropy [2] as another generalization of Shannon entropy. For a continuous and strictly monotonic function $\phi$ on $(0,1]$, the quasilinear entropy is given by

[TABLE]

If we take $\phi(x)=\log x$ in (4), then I_{1}^{\log}\big{(}\mathbf{p}\big{)}=H_{1}(\mathbf{p}). We may also redefine the quasilinear entropy by

[TABLE]

for a continuous and strictly monotonic function $\psi$ on \big{(}0,\infty\big{)}. If we take $\psi(x)=\log x$ in (5), then we have $I_{1}^{\log}(\mathbf{p})=H_{1}(\mathbf{p})$. The case $\psi(x)=x^{1-q}$ is also useful in practice, since we recapture the Rényi entropy, namely I_{1}^{x^{1-q}}\big{(}\mathbf{p}\big{)}=R_{q}(\mathbf{p}), where the Rényi entropy [3] is defined by

[TABLE]

The generalized entropies involving Tsallis entropies and quasilinear entropies were stuided in [19] by the use of refined Young inequality.

Definition 1.0.1

For a continuous and strictly monotonic function $\psi$ on $(0,\infty)$ and two probability distributions $\mathbf{p}=\{p_{1},\cdots,p_{n}\}$ and $\mathbf{r}=\{r_{1},\cdots,r_{n}\}$ with $p_{j}>0,r_{j}>0,$ for all $j=1,\cdots,n$, the quasilinear divergence is defined by

[TABLE]

The quasilinear divergence coincides with the usual divergence if $\psi(x)=\log x$, i.e.,

[TABLE]

We denote by $D_{q}^{R}(\mathbf{p}||\mathbf{r})$ the Rényi divergence [1] defined by

[TABLE]

This is another particular case of the quasilinear divergence, namely, for \psi\big{(}x\big{)}=x^{1-q}, we have

[TABLE]

From [1], we denote by

[TABLE]

the Tsallis divergence, which can be written with $q$-logarithm as follows:

[TABLE]

The Tsallis divergence converges to the usual divergence (relative entropy, Kullback-Leibler information) as $q\to 1$:

[TABLE]

Another divergence (relative entropy) is called $\alpha$-divergence [1], given by

[TABLE]

for $\alpha\neq\pm 1$. We recall from [1] that

[TABLE]

and

[TABLE]

where $1\neq q>0$. Using the inequality $x\geq\ln(1+x)$, for every $x>-1$, we deduce that, for $0<q<1$ we have

[TABLE]

and for $q>1$ we have

[TABLE]

Recall the following definition:

Definition 1.0.2

([16])* For a continuous and strictly monotonic function $\psi$ on $(0,\infty)$ and $q>0$ with $q\neq 1$, the Tsallis quasilinear entropy ($q$-quasilinear entropy) is defined by*

[TABLE]

where $\mathbf{p}=\{p_{1},\cdots,p_{n}\}$ is a probability distribution with $p_{j}>0$ for all $j=1,\cdots,n.$

Notice that if $\psi$ does not depend on the parameter $q$, then we have

[TABLE]

For $x>0$ and $q>0$ with $q\neq 1$, we define the $q$-exponential function as the inverse function of the $q$-logarithmic function by

[TABLE]

Note that the function $\exp_{q}(x)$ is the solution of the differential equation $\frac{dy}{dx}=y^{q}$ [18], where $y(0)=1$ and $q\in\mathbb{R}.$

If we take $\psi(x)=\ln_{q}x$, then we have $I_{q}^{\ln_{q}}(\mathbf{p})=H_{q}(\mathbf{p}).$ Furthermore, we have $I_{q}^{x^{1-q}}(\mathbf{p})=H_{q}(\mathbf{p}).$

Proposition 1.0.3

([16])* The Tsallis quasilinear entropy is nonnegative:*

[TABLE]

We note here that the $q$-exponential function gives us the following connection between Rényi entropy and Tsallis entropy [4]:

[TABLE]

We should note here that \exp_{q}H_{q}\big{(}\mathbf{p}\big{)} is always defined, since we have

[TABLE]

Definition 1.0.4

For a continuous and strictly monotonic function $\psi$ on $(0,\infty)$ and two probability distributions $\mathbf{p}=\{p_{1},\cdots,p_{n}\}$ and $\mathbf{r}=\{r_{1},\cdots,r_{n}\}$ with $p_{j}>0,r_{j}>0,$ for all $j=1,\cdots,n$, the Tsallis quasilinear divergence is defined by

[TABLE]

We notice that if $\psi$ does not depend on the parameter $q$. We have

[TABLE]

For $\psi(x)=\ln_{q}x$, the Tsallis quasilinear divergence becomes Tsallis divergence,

[TABLE]

And for $\psi(x)=x^{1-q}$, we have

[TABLE]

Proposition 1.0.5

([16])* If $\psi$ is a concave increasing function or a convex decreasing function, then we have nonnegativity of the Tsallis quasilinear divergence:*

[TABLE]

Remark 1.0.6

The following two functions satisfy the sufficient condition in the above proposition:

• (i)

$\psi(x)=\ln_{q}(x)$* for $q>0$ with $q\neq 1$.*

• (ii)

$\psi(x)=x^{1-q}$* for $q>0$ with $q\neq 1$.*

It is notable that the following identity holds:

[TABLE]

2 New bounds for Tsallis quasilinear entropy and divergence

We start with the following lemma.

Lemma 2.0.1

Let $q\neq 1$ be a real number strictly positive.

• (I)

Let $0<x\leq 1$.

• (I-i)

If $0<q<1$, then

[TABLE]

• (I-ii)

If $q>1$, then

[TABLE]

• (II)

Let $x\geq 1$.

• (II-i)

If $0<q<1$, then

[TABLE]

• (II-ii)

If $q>1$, then

[TABLE]

Proof. For the case of $x=1$, all inequalities (21), (22),(23) and (24) hold trivially. So we assume $x\neq 1$ and $x>0$ in the sequel. We use the following identity [20, Theorem 2.3],

[TABLE]

with $q\neq 1$, $x\neq 1$ and $x>0$.

We also use the Hermite-Hadamard inequality for the convex function $f(t)$:

[TABLE]

Since the function $f_{q}(t)=x^{(1-q)t}$ on $t\in[0,1]$ is convex for $x>0,x\neq 1,q>0$ by $\frac{d^{2}f_{q}(t)}{dt^{2}}=(1-q)^{2}x^{(1-q)t}(\log x)^{2}\geq 0$, we have

[TABLE]

• (I)

If $0<x<1$, then

[TABLE]

which shows the second and third inequalities in (21) and (22).

• (I-i)

If $0<q<1$, then we have the first and last inequalities in (21) since $\frac{x^{1-q}+1}{2}\leq 1$ and $x^{1-q}\leq x^{(1-q)/2}$.

• (I-ii)

If $q>1$, then we have the first and last inequalities in (22) since $\frac{x^{1-q}+1}{2}\leq x^{1-q}$ and $1\leq x^{(1-q)/2}$.

• (II)

If $x>1$, then

[TABLE]

which shows the second and third inequalities in (23) and (24).

• (II-i)

If $0<q<1$, then we have the first and last inequalities in (23) since $1\leq x^{(1-q)/2}$ and $\frac{x^{1-q}+1}{2}\leq x^{1-q}$.

• (II-ii)

If $q>1$, then we have the first and last inequalities in (24) since $x^{1-q}\leq x^{(1-q)/2}$ and $\frac{x^{1-q}+1}{2}\leq 1$.

∎

To state the following proposition, we recall the quasi-entropy:

[TABLE]

which appeared in [2, Eq.(7.1.1)] as a special case. See also [17, Result 10.15.]. We have the following results as a simple consequence of Lemma 2.0.1.

Proposition 2.0.2

Let $q\neq 1$ be a real number strictly positive and $\textbf{p}=\{p_{1},\cdots,p_{n}\}$ a probability distribution with $p_{j}>0$ for all $j=1,\cdots,n$. If $q>1$, then we have

[TABLE]

If $0<q<1$, then we have

[TABLE]

Theorem 2.0.3

Let $\psi$ be a continuous and strictly monotonic function on $(0,\infty)$, $q>0$ with $q\neq 1$, and let $\textbf{p}=\{p_{1},\cdots,p_{n}\}$ be a probability distribution with $p_{j}>0$ for all $j=1,\cdots,n$. If $0<q<1$, then we have

[TABLE]

and if $q>1$, then we have

[TABLE]

where $\frac{1}{\mathbf{p}}$ means $\left\{\frac{1}{p_{1}},\cdots,\frac{1}{p_{n}}\right\}$.

Proof. If the function $\psi$ is strictly increasing, then the function $\psi^{-1}$ is strictly increasing. Since $\frac{1}{p_{j}}>1,$ for every $j\in\{1,...,n\}$, we deduce $\psi\left(\frac{1}{p_{j}}\right)>\psi\left(1\right).$ Therefore, we have $\sum_{j=1}^{n}p_{j}\psi\left(\frac{1}{p_{j}}\right)>\psi\left(1\right)\sum_{j=1}^{n}p_{j}=\psi\left(1\right)$. It implies that $\psi^{-1}\left(\sum_{j=1}^{n}p_{j}\psi\left(\frac{1}{p_{j}}\right)\right)>1$. Similarly it is proven for a strictly decreasing function $\psi$.

If $0<q<1$, then from Lemma 2.0.1, for $x=\psi^{-1}\left(\sum_{j=1}^{n}p_{j}\psi\left(\frac{1}{p_{j}}\right)\right)>1$, we have

[TABLE]

which imply inequalities (30). Similarly we deduce the reversed inequalities (31).

∎

Corollary 2.0.4

Let $\psi$ be a continuous and strictly monotonic function on $(0,\infty)$, $q>0$ with $q\neq 1$ and let $\textbf{p}=\{p_{1},\cdots,p_{n}\}$ be a probability distribution with $p_{j}>0$ for all $j=1,\cdots,n$. If $0<q<1$, then we have

[TABLE]

and if $q>1$, then we have

[TABLE]

Proof. For $\psi(x)=\log x$ from Theorem 2.0.3 it follows that we have the inequalities of the statement, taking into account of $I_{1}^{\log}(\mathbf{p})=H(\mathbf{p})$ and $I_{q}^{\log}\left(\mathbf{p}\right)=\frac{1}{1-q}\left(\exp\left((1-q)H(\mathbf{p})\right)-1\right).$

∎

Corollary 2.0.5

Under the same assumptions as in Corollary 2.0.4, for $0<q<1$ we have

[TABLE]

and if $q>1$, then we have

[TABLE]

Proof. For \psi\big{(}x\big{)}=x^{1-q} from Theorem 2.0.3 it follows that we have the relations of the statement.

∎

Next, we obtain an estimation for the Tsallis quasilinear divergence.

Theorem 2.0.6

Let $\psi:I\to J,J\subseteq(0,\infty)$ be a concave increasing function or a convex decreasing function, let $\textbf{p}=\{p_{1},\cdots,p_{n}\}$ and $\textbf{r}=\{r_{1},\cdots,r_{n}\}$ be two probability distributions with $p_{j}>0,r_{j}>0$ for all $j=1,\cdots,n$. If $q>1$, then we have

[TABLE]

If $0<q<1$, then we have

[TABLE]

Proof. We firstly assume that $\psi$ is a concave increasing function. From the Jensen’s inequality

[TABLE]

which is equivalent to

[TABLE]

In this case, $\psi^{-1}$ is also increasing, so we have

[TABLE]

If $q>1$, then from Lemma 2.0.1 we have

[TABLE]

which is equivalent to the inequalities in (32).

In the case that $\psi$ is a convex decreasing function, we can similarly prove the inequalities in (36). By the similar way, we prove the inequalities in (37) in the case $0<q<1$.

∎

3 Biparametrical extended entropies and divergences

In this section, we consider two different kinds of biparametrical extended entropies and divergences. Firstly we give bounds on these defined in [21, 1] in Subsection 3.1. Secondly, we give bounds for the $(r,q)$-quasilinear entropy and divergence in Subsection 3.2.

3.1 Biparametrical extended entropy and divergence given in [1, 21]

We recall that Wada and Suyari in [21] have axiomatically defined the biparametrical extended entropy by

[TABLE]

for $q,r\in\mathbb{R}$ such that $0\leq q\leq 1\leq r$ or $0\leq r\leq 1\leq q$, where $\mathbf{p}=\{p_{1},\cdots,p_{n}\}$ is a probability distribution. Since the equality $\ln_{q-r+1}x=\frac{x^{r-q}-1}{r-q}$ holds, multiplying by $-x^{q}$, we deduce the relation

[TABLE]

where $q+1\geq r$ and $q\neq r$. As a consequence,

[TABLE]

It is easy to see that

[TABLE]

For $r=1$ and $q\neq 1$ from (39) we have

[TABLE]

Consequently

[TABLE]

In [1], the relation between the biparametrical extended entropy and the Tsallis entropy, which is expressed by a convex combination, was given by:

[TABLE]

Proposition 3.1.1

Let $q,r\neq 1$ be two real numbers strictly positive and $\textbf{p}=\{p_{1},\cdots,p_{n}\}$ a probability distribution with $p_{j}>0$ for all $j=1,\cdots,n$. If $q>1$ and $0<r<1$ then we have

[TABLE]

and if $0<q<1$ and $r>1$, then we have

[TABLE]

Proof. If $q>1$ and $0<r<1$, then using inequalities (28) and (29), we have

[TABLE]

and

[TABLE]

Using these inequalities with the identity (40), we obtain the inequalities (41) and (40). ∎

Remark 3.1.2

The inequalities from Proposition (41) can be similarly proven by swapping $r$ and $q$ in (40), (28) and (29), with $S_{r,q}({\bf p})=S_{q,r}({\bf p})$.

In [1], we established that the biparametrical extended divergence was axiomatically given by

[TABLE]

for two real numbers $q$ and $r$ such that $q\neq r$, where $\mathbf{p}=\{p_{1},\cdots,p_{n}\}$ and $\mathbf{r}=\{r_{1},\cdots,r_{n}\}$ are two probability distributions. The biparametrical extended divergence is a generalization of the Tsallis divergence, because for $r=1$ in (43), we deduce the following identity:

[TABLE]

Moreover, we have

[TABLE]

and a convex combination between the biparametrical extended divergence and the Tsallis divergence expressed by

[TABLE]

This divergence is nonnegative, i.e. $D_{q,r}(\mathbf{p}||\mathbf{r})\geq 0$, and has many other properties: symmetry, joint convexity, monotonicity.

Next, we define the quasi-divergence:

[TABLE]

Notice that the quasi-divergence $D_{(q)}(\mathbf{p}||\mathbf{r})$ is a generalization of the divergence, since for $q=1$ we obtain $D_{(1)}(\mathbf{p}||\mathbf{r})=D_{1}(\mathbf{p}||\mathbf{r})$.

Lemma 3.1.3

Let $q\neq 1$ be a real number strictly positive and $\textbf{p}=\{p_{1},\cdots,p_{n}\}$ and $\textbf{r}=\{r_{1},\cdots,r_{n}\}$ two probability distributions with $p_{j}>0,r_{j}>0$ for all $j=1,\cdots,n$. If $q>1$ then we have

[TABLE]

and if $0<q<1$, then we have

[TABLE]

Proof. If $q>1$, then from the proof of Lemma 2.0.1, we have the following inequality:

[TABLE]

for all $x>0$. Consequently for $x=\frac{r_{j}}{p_{j}}$, we obtain

[TABLE]

Multiplying by $p_{j}$ and passing to the sum, we deduce

[TABLE]

which implies the statement. Similarly we prove for the case $0<q<1$.

∎

Using the identity (44) with the inequalities (45) and (46), we obtain the following results.

Proposition 3.1.4

Let $q,r\neq 1$ be strictly positive real numbers and $\textbf{p}=\{p_{1},\cdots,p_{n}\}$ and $\textbf{r}=\{r_{1},\cdots,r_{n}\}$ two be probability distributions with $p_{j}>0,r_{j}>0$ for all $j=1,\cdots,n$. If $r>1$ and $0<q<1$, then we have

[TABLE]

and if $0<r<1$ and $q>1$, then we have

[TABLE]

3.2 A biparametrical extended entropy and divergence defined by the $(r,q)$-logarithmic function

We firstly give the notation. The biparametrical extended logarithmic function (see e.g. [19]) for $x>0$ is defined by

[TABLE]

which uniformly converges to the usual logarithmic function $\log x$ as $q\to 1$ and $r\to 1$. This is a decreasing function with respect to the indices. Correspondingly, the inverse function of $\ln_{r,q}x$ is denoted by

[TABLE]

We start with the Tsallis $(r,q)$-quasilinear entropies and Tsallis $(r,q)$-quasilinear divergences as they were defined in [15].

Definition 3.2.1

Let $\psi$ be a continuous and strictly monotonic function on $(0,\infty)$ and $q,r>0$ with $q,r\neq 1$. The $(r,q)$-quasilinear entropy is defined by

[TABLE]

For $\psi(x)=\ln_{r,q}x$ we have the following entropic functional:

[TABLE]

This also gives rise to another case of interest:

[TABLE]

which in particular coincides with Arimoto entropy.

Definition 3.2.2

For a continuous and strictly monotonic function $\psi$ on $(0,\infty)$ and $q,r>0$ with $q,r\neq 1$ and two probability distributions $\mathbf{p}=\{p_{1},\cdots,p_{n}\}$ and $\mathbf{r}=\{r_{1},\cdots,r_{n}\}$ with $p_{j}>0,r_{j}>0$ for all $j=1,\cdots,n$, the $(r,q)$-quasilinear divergence is defined by

[TABLE]

For $\psi(x)=\ln_{r,q}x$ we have the following relation:

[TABLE]

By a direct calculation we have

[TABLE]

and

[TABLE]

Thus

• (i)

If $q\leq 1$ and $r\leq 0$, then $\frac{d^{2}\ln_{r,q}x}{dx^{2}}\geq 0$.

• (ii)

If $q\geq 1$ and $r\geq 0$, then $\frac{d^{2}\ln_{r,q}x}{dx^{2}}\leq 0$.

Therefore, we obtain the non-negativity of the biparametrical divergence

[TABLE]

for $q\geq 1$ and $r\geq 0$, by using Jensen’s inequality.

By analogy to the entropy computation, the following Arimoto type divergence:

[TABLE]

The non-negativity in the above inequality follows from the fact that $p_{j}^{r}r_{j}^{1-r}\leq rp_{j}+(1-r)r_{j}$ for $r\in[0,1]$ and for each $j=1,\cdots,n$, and its reverse holds for $r\notin[0,1]$ and for each $j=1,\cdots,n$.

Similarly, we apply Lemma 2.0.1 for the biparametric case. Above we defined the \big{(}r,q\big{)}-logarithmic function for $x>0$ by

[TABLE]

Lemma 3.2.3

Let $q,r\neq 1$ be two strictly positive real numbers. If $0<x\leq 1$, $q>1$, then we have

[TABLE]

If $0<x\leq 1$, $0<q<1$, then we have

[TABLE]

If $x\geq 1$, $0<q<1$, then we have

[TABLE]

If $x\geq 1$, $q>1$, then we have

[TABLE]

Proof. For $r\neq 1$ and $0<x\leq 1$ we have $0<\exp\left(\ln_{r}x\right)\leq 1$. Using Lemma 2.0.1 for $q>1$ and $x=\exp(\ln_{r}x)$ we have

[TABLE]

which implies inequality (3.2.3). Similarly, we show the other cases.

∎

Theorem 3.2.4

Let $q,r\neq 1$ be two strictly positive real numbers. Let $\psi$ be a continuous and strictly monotonic function on $(0,\infty)$ and $\textbf{p}=\{p_{1},\cdots,p_{n}\}$ a probability distribution with $p_{j}>0$ for all $j=1,\cdots,n$. If $0<q<1$, then we have

[TABLE]

and if $q>1$, then we have

[TABLE]

Proof. If $0<q<1$ and $x=\psi^{-1}\left(\sum_{j=1}^{n}p_{j}\psi\left(\frac{1}{p_{j}}\right)\right)>1$, then, using inequality (3.2.3) we deduce the statement. Similarly we deduce the reversed inequalities using inequality (3.2.3).

∎

Theorem 3.2.5

Let $q,r\neq 1$ be two strictly positive real numbers. Let $\psi:I\to J,J\subseteq(0,\infty)$ be a concave increasing function or a convex decreasing function, $\textbf{p}=\{p_{1},\cdots,p_{n}\}$ and $\textbf{r}=\{r_{1},\cdots,r_{n}\}$ be two probability distributions with $p_{j}>0,r_{j}>0$ for all $j=1,\cdots,n$. If $q>1$, then

[TABLE]

If $0<q<1$, then

[TABLE]

Proof. From the proof of Theorem 2.0.6, we deduce

[TABLE]

If $q>1$, then from inequality (3.2.3) we obtain the inequalities (57). For the case that $\psi$ is a convex decreasing function, we can similarly prove the inequality. Similarly we prove the case $0<q<1$.

∎

Remark 3.2.6

It is well known that $\exp(x)\geq 1+x$ for every $x\in\mathbb{R}$. For $x=\frac{\left(1-q\right)\left(\left(\frac{1}{p_{j}}\right)^{1-r}-1\right)}{1-r}$ we have, when $0<q<1$, the inequality

[TABLE]

which implies that $\ln_{r,q}\frac{1}{p_{j}}\geq\ln_{r}\frac{1}{p_{j}}.$ Multiplying by $p_{j}$ and passing to the sum from $1$ to $n$, we obtain for $0<q<1$,

[TABLE]

An analogous way, for $q>1$, we deduce

[TABLE]

Using the above inequality, for $0<q<1$, we have $\ln_{r,q}\frac{r_{j}}{p_{j}}\geq\ln_{r}\frac{r_{j}}{p_{j}}.$ Multiplying by $-p_{j}$ and passing to the sum from $1$ to $n$, we obtain for $0<q<1$,

[TABLE]

and for $q>1$ we have

[TABLE]

4 Some inequalities for the extended Lin’s divergence

The Tsallis divergence (relative entropy) is rewritten by

[TABLE]

The Jeffreys divergence is defined by

[TABLE]

and the Jensen-Shannon divergence is defined as

[TABLE]

(see e.g. [19]).

In [22, Lemma 7] we proved the general case of following inequality with a parameter $\lambda$ in hypodivergence.

[TABLE]

We can prove the following inequality with one parameter $0<v\leq 1$.

Theorem 4.0.1

Let $v\in(0,1]$ and let $p_{j}>0$, $r_{j}>0$ with $\sum_{j=1}^{n}p_{j}=\sum_{j=1}^{n}r_{j}=1$ for two probability distributions ${\bf p}=\left\{p_{1},\cdots,p_{n}\right\}$ and ${\bf r}=\left\{r_{1},\cdots,r_{n}\right\}$. Then for $q\in(0,1)$ we have

[TABLE]

and for $q>1$ we deduce the inverse inequality.

Proof. Using the arithmetic-geometric mean inequality, we have

[TABLE]

Similarly, using the geometric-harmonic mean inequality, we have

[TABLE]

which implies the second inequality.

∎

Note that the first inequality in Theorem 4.0.1 recovers the inequality (64) when $v=1/2$.

Lin’s divergence [23] is given by

[TABLE]

From inequality (64) by passing to the limit when $q\to 1$, we have the inequality

[TABLE]

Similarly, we have

[TABLE]

By summing the above relations, we obtain an inequality between the Jeffreys divergence and Jensen-Shannon divergence:

[TABLE]

Proposition 4.0.2

Fro two probability distributions ${\bf p}=\{p_{1},\cdots,p_{n}\}$ and ${\bf r}=\{r_{1},\cdots,r_{n}\}$, we have

[TABLE]

Proof. Using the Lin’s divergence and the usual divergence, we have

[TABLE]

Consider the function $f:(0,\infty)\to\mathbb{R}$ given by $f(x)=(1-x)\log\frac{2x}{1+x}$. The function $f$ is concave, because $\frac{d^{2}f(x)}{dx^{2}}=\frac{-3x-1}{x^{2}(x+1)^{2}}\leq 0$. Therefore, applying Jensen’s inequality, we obtain

[TABLE]

Consequently, then follows the inequality of the statement. ∎

5 Some characterizations of the Fermi-Dirac entropy and the Bose-Einstein entropy

In [24] the physical phenomena for power-law were studied from Tsallis statistical viewpoints using the Fermi$-$Dirac$-$Tsallis entropy given by

[TABLE]

Similarly, in [15] the Bose$-$Einstein$-$Tsallis entropy is defined as

[TABLE]

These entropies are one-parameter extensions of the Fermi-Dirac entropy and the Bose-Einstein entropy defined by

[TABLE]

and

[TABLE]

respectively.

Theorem 5.0.1

Let $\mathbf{p}=\{p_{1},\cdots,p_{n}\}$ be a probability distribution satisfying $p_{j}>0$ for all $j=1,\cdots,n$. Then we have

[TABLE]

Proof: Mention that $l_{1}^{BE}(\textbf{p})-l_{1}^{FD}(\textbf{p})=\sum_{j=1}^{n}\left(\left(1-p_{j}\right)\log\left(1-p_{j}\right)+\left(1+p_{j}\right)\log\left(1+p_{j}\right)\right)$. We consider the function $f:\left[0,1\right)\to\mathbb{R}$, defined by $f\left(x\right)=\left(1-x\right)\log\left(1-x\right)+\left(x+1\right)\log\left(x+1\right).$ Since $\frac{df\left(x\right)}{dx}=\log\frac{1+x}{1-x}>0$, we deduce that the function $f$ is increasing. So, $f\left(x\right)\geq f\left(0\right)=0,$ which means that $\sum_{j=1}^{n}\left(\left(1-p_{j}\right)\log\left(1-p_{j}\right)+\left(1+p_{j}\right)\log\left(1+p_{j}\right)\right)\geq 0.$ Therefore, we obtain the inequality of the statement. ∎

It may be of interest for the readers to give the following alternative proof of Theorem 5.0.1. Alternative proof of Theorem 5.0.1: For a probability distribution $\mathbf{p}=\{p_{1},\cdots,p_{n}\}$ with $p_{j}>0$, for all $j=1,\cdots,n$, we deduce two probability distributions $\mathbf{p^{\prime}}=\{p_{1}^{\prime},\cdots,p_{n}^{\prime}\}$, with $p_{j}^{\prime}=\frac{1-p_{j}}{n-1}>0$, for all $j=1,\cdots,n$, and $\mathbf{p"}=\{p_{1}",\cdots,p_{n}"\}$, with $p_{j}"=\frac{1+p_{j}}{n+1}>0$, for all $j=1,\cdots,n$. It is easy to see that

[TABLE]

and

[TABLE]

where $n\geq 2.$

Taking the difference between the two above relations, we have the following relation:

[TABLE]

But, for a probability distribution $\mathbf{p}=\{p_{1},\cdots,p_{n}\}$ we generally have $0\leq H(\mathbf{p})\leq\log n$, so we deduce $H(\mathbf{p^{\prime}})\leq\log n$ and $H(\mathbf{p"})\leq\log n$. Therefore, we find that

[TABLE]

The last inequality can be proven by $\frac{n^{2n}}{\left(n-1\right)^{n-1}\left(n+1\right)^{n+1}}\leq\frac{n}{n+1}\leq 1$. This first inequality is equivalent to $a_{n-1}\geq a_{n}$, where the sequence $a_{n}$ is given by $a_{n}=\left(1+\frac{1}{n}\right)^{n}$ and it is increasing for $n\geq 1.$ ∎

Similarly we can prove the following result.

Theorem 5.0.2

For a probability distribution $\mathbf{p}=\{p_{1},\cdots,p_{n}\}$ with $p_{j}>0$ for all $j=1,\cdots,n$ and $r\in\mathbb{R}$, we have

[TABLE]

Proof. The special case $r=1$ was proven in Theorem 5.0.1. In the sequel we assume $r\neq 1$. Since

[TABLE]

we consider the function $f_{r}:\left[0,1\right)\to\mathbb{R}$, defined by $f_{r}\left(x\right)=2-\left(1+x\right)^{r}-\left(1-x\right)^{r}.$ We find $\frac{df_{r}\left(x\right)}{dx}=r\left\{\left(1-x\right)^{r-1}-\left(1+x\right)^{r-1}\right\}$. For $r>1$ and $0<x<1$, we have $\left(1-x\right)^{r-1}-\left(1+x\right)^{r-1}<0,$ which proves that $\frac{df_{r}\left(x\right)}{dx}<0$. Therefore, the function $f_{r}$ is decreasing, so $f_{r}\left(x\right)\leq f_{r}\left(0\right)=0,$ which means that

[TABLE]

But, since $1-r<0$, we obtain $\frac{1}{1-r}\sum_{j=1}^{n}\left\{2-\left(1+p_{j}\right)^{r}-\left(1-p_{j}\right)^{r}\right\}\geq 0$, which shows that the inequality of the statement is true. An analogous way, for $r<1$, we deduce the inequality of the statement. ∎

Remark 5.0.3

The Fermi$-$Dirac$-$Tsallis entropy $l_{r}^{FD}(\textbf{p})$ converges to the Fermi$-$Dirac entropy $l_{1}^{FD}(\textbf{p})$, and the Bose$-$Einstein$-$Tsallis entropy $l_{r}^{FD}(\textbf{p})$ converges to the Bose$-$Einstein entropy $l_{1}^{FD}(\textbf{p})$, when we take the limit $q\to 1$. Therefore, by passing to the limit in inequality (71), when $r\to 1$, we obtain inequality (70).

6 Conclusion

In this paper, we have obtained some mathematical inequalities for some entropies and divergences. In section 3, we studied some mathematical properties on the biparametrical extended entropy $H_{r,q}({\bf p})$ given in (50) and $S_{r,q}({\bf p})$ in (38). Also we found the biparametrical extended divergences in the same section to be interested as $D_{r,q}({\bf p}||{\bf r})$ given in (52) and $\hat{D}_{r,q}({\bf p}||{\bf r})$ given in (43). It is also natural to be interested in the relations between them. We easily find that

[TABLE]

and

[TABLE]

Since it is quite difficult to find the relation for any parameters $q$ and $r$, we will try to study about it in the future.

Acknowledgements

The authors would like to thank the referees for their careful and insightful comments to improve our manuscript. The author (S.F.) was partially supported by JSPS KAKENHI Grant Number 16K05257.