Route from discreteness to the continuum for the non-logarithmic $q$-entropy

TL;DR

This paper demonstrates that the discrete nonlogarithmic q-entropy converges to a well-defined continuous form, establishing its relevance for classical systems and linking it to known relative entropy measures.

Contribution

It proves the continuum limit of the nonlogarithmic q-entropy and connects it to the Csiszár type q-relative entropy, resolving a key open problem.

Findings

Discrete q-entropy converges to a continuous form.

Negative q-entropy with continuous variables relates to q-relative entropy.

No fundamental obstacle exists for applying q-entropy to continuous systems.

Abstract

The existence and exact form of the continuum expression of the discrete nonlogarithmic $q$-entropy is an important open problem in generalized thermostatistics, since its possible lack implies that nonlogarithmic $q$-entropy is irrelevant for the continuous classical systems. In this work, we show how the discrete nonlogarithmic $q$-entropy in fact converges in the continuous limit and the negative of the $q$-entropy with continuous variables is demonstrated to lead to the (Csisz{\'a}r type) $q$-relative entropy just as the relation between the continuous Boltzmann-Gibbs expression and the Kullback-Leibler relative entropy. As a result, we conclude that there is no obstacle for the applicability of the $q$-entropy to the continuous classical physical systems.

Equations

$$S(\{p\})=\sum _{i=1}^n{p}_i\ln (1/p_i)$$

$$S(\rho )=\underset{a}{\overset{b}{\int }}\rho (x)\ln \left(\frac{1}{\rho (x)}\right)\mathrm{d}x$$

$$p_i=\rho (x_i)\Delta x_i,\Delta x_i=\frac{1}{nm(x_i)}$$

$$\sum _{i=1}^n\rho (x_i)\Delta x_i=1\stackrel{n\to \infty }{⟶}{\int }_a^b\rho (x)\mathrm{d}x=1.$$

$$S(\{p\})=\sum _{i=1}^n{p}_i\ln \left(\frac{m(x_i)}{\rho (x_i)}\right)+\ln (n)$$

$$S(\{p\})=\sum _{i=1}^n\rho (x_i)\ln \left(\frac{m(x_i)}{\rho (x_i)}\right)\Delta x_i+\ln (n)$$

$$\underset{n\to \infty }{\lim }S(\{p\})=S(\rho )=\underset{a}{\overset{b}{\int }}\rho (x)\ln \left(\frac{m(x)}{\rho (x)}\right)\mathrm{d}x,$$

$$S_q(\{p\})=n^{q-1}\sum _{i=1}^n{p}_i{\ln }_q\left(1/p_i\right),$$

$${\ln }_q(x)\equiv \frac{x^{1-q}-1}{1-q},$$

$$S_q(\{p\})=n^{q-1}\sum _{i=1}^n{p}_i{\ln }_q\left(\frac{n{m}_q(x_i)}{\rho (x_i)}\right)=n^{q-1}\left[\sum _{i=1}^n{p}_i{\ln }_q\left(\frac{m_q(x_i)}{\rho (x_i)}\right)-{\ln }_q\left(1/n\right)\sum _{i=1}^n{p}_i^q\right].$$

$$\sum _{i=1}^n{p}_i^q=[1+(1-q)n^{1-q}{S}_q].$$

$$S_q(\{p\})=\sum _{i=1}^n\rho (x_i){\ln }_q\left(\frac{m_q(x_i)}{\rho (x_i)}\right)\Delta x_i+{\ln }_{2-q}\left(n\right).$$

$$S_q\left(\rho \right)=\underset{n\to \infty }{\lim }{S}_q(\{p\})=\underset{a}{\overset{b}{\int }}\rho (x){\ln }_q\left(\frac{m_q(x)}{\rho (x)}\right)\mathrm{d}x.$$

$$K\left[\rho \Vert m\right]=\underset{a}{\overset{b}{\int }}\rho (x)\ln \left(\frac{\rho (x)}{m(x)}\right)\mathrm{d}x,$$

$$-S_q\left(\rho \right)=-\underset{a}{\overset{b}{\int }}\rho (x){\ln }_q\left(\frac{m_q(x)}{\rho (x)}\right)\mathrm{d}x=\underset{a}{\overset{b}{\int }}\rho (x){\ln }_{2-q}\left(\frac{\rho (x)}{m_q(x)}\right)\mathrm{d}x,$$

$$K\left[p\Vert 1/n\right]=\sum _{i=1}^n{p}_i\ln \left(\frac{p_i}{r_i}\right)=\sum _{i=1}^n{p}_i\ln \left(n{p}_i\right)=-S(\{p\})-\ln (1/n)$$

$$K_q\left[p\Vert 1/n\right]=-n^{q-1}\sum _{i=1}^n{p}_i{\ln }_q(1/p_i)-{\ln }_q\left(1/n\right)=-S_q(\{p\})-{\ln }_q\left(1/n\right)$$

Full text

LABEL:FirstPage1 LABEL:LastPage#11

Since its advent, the nonadditive $q$-entropy Tsallis88 ; Tsallisbook has found numerous fields of application in many diverse fields Chang ; Tribeche ; Rastegin ; bath ; Portesi ; Campisi ; Rotundo ; Van1 ; high2 ; high3 ; third . Despite this apparent progress in the field, however, there have been some criticisms regarding its applicability and scope. Among such criticisms, one can particularly cite the ones related to the Bayesian updating procedure Presse , Lesche stability Lesche ; Abe1 ; Lutsko , and the methodology of the entropy maximization OikBagci2010 .

Recently, Abe pinpointed that the nonadditive $q$-entropy is inherently limited to the finite discrete systems, since its continuum expression has not been obtained yet Abecomment (see also Refs. Andresen ; Abereply ). In this work, we show that one can indeed obtain the concomitant continuum expressions of the nonadditive entropy and therefore point out that the nonadditive $q$-entropy can also be used for continuous physical systems.

Before proceeding further with the nonadditive case, one should be convinced why taking the route from discreteness to a continuum is essential concerning any entropy measure in general. Setting the Boltzmann constant to unity, the finite discrete Boltzmann-Gibbs (BG) entropy reads

[TABLE]

where $p_{i}$ denotes the probability of the $i$th event. Let us now consider its continuous counterpart to be the following expression

[TABLE]

where $\rho(x)$ is a probability density function satisfying the normalization condition in the interval $[a,b]$.

Although the continuous expression above seems reasonable at first sight, it has three serious drawbacks. First, the continuous version in Eq. (2) has an overall unit of $\log($length$)$ whereas the discrete entropy in Eq. (1) is dimensionless Uffink . Second, the probability density $S(\rho)$ is not invariant with respect to coordinate transformations Uffink . Last but not the least, the discrete BG entropy $S(\{p\})$ in the $n\rightarrow\infty$ limit and $S(\rho)$ yield different results Uffink : To see this more explicitly, consider a uniform distribution $\rho(x)$ in the interval $[a,b]$ as $1/(b-a)$ so that its discrete counterpart $p(x_{i})$ is given by $1/n$ obtained through dividing the same interval $[a,b]$ into $n$ equal subintervals where the index $i$ runs from $1$ to $n$. Then, the continuous entropy $S(\rho)$ for this uniform distribution yields $\ln(b-a)$ while the discrete expression $S(\{p\})$ attains infinity in the $n\rightarrow\infty$ limit. In other words, the continuum version of the discrete entropy does not converge to the value obtained through the continuous version for the uniform distribution. Therefore, the continuous version of the discrete BG entropy $S(\{p\})$ cannot be $S(\rho)$.

The solution of the discrete-to-continuum transition for the BG entropy is already known Jaynes . In order to extend BG entropy to the continuum, we assume some discrete points $x_{i}$ with $i=1,2,\ldots,n$ and $x_{1}<\cdots<x_{n}$ filling the interval $[a,b]$ so that one has a factorizable discrete probability $p_{i}$ Jaynes as

[TABLE]

with the property

[TABLE]

Substitution of Eq. (3) into the discrete entropy expression given by Eq. (1) yields

[TABLE]

where we have also made use of the normalization $\sum_{i=1}^{n}p_{i}=1$. Equation (5) can now be rewritten as

[TABLE]

so that the above summation in the $n\rightarrow\infty$ limit finally yields the following continuous expression

[TABLE]

where the additive divergent term $\lim_{n\rightarrow\infty}\ln\left(n\right)$ is omitted since the entropy is not absolute, but only its change can be measured Abecomment . It is worth remarking that the continuous entropy expression given by Eq. (7) is dimensionless like its discrete counterpart and invariant under different reparametrization of continuum.

Note that the usual discrete nonadditive $q$-entropy, i.e., $S_{q}=\sum_{i=1}^{n}p_{i}\ln_{q}(1/p_{i})$ Tsallis88 ; Tsallisbook ($\ln_{q}(x)$ is defined in Eq. (9)), cannot be adopted, since it does not converge in the continuous limit Abecomment . Therefore, we consider

[TABLE]

where the $q$-logarithm OikBagci2009 is defined as

[TABLE]

which becomes the ordinary logarithm in the $q\rightarrow 1$ limit so that the nonadditive entropy becomes the BG entropy. The discrete entropy expression in Eq. (8) has an additional multiplicative term $n^{q-1}$ compared to the usual nonadditive entropy expression Tsallis88 ; Tsallisbook . As we show below, this term is required for convergence and therefore can be called the convergence factor (see Eq. (17) below for more on its justification).

In order to extend the discrete expression above to the continuum, we consider the same apparatus as before (see Eq. (3) and related explanations above it) with the exception that we now have $\Delta x_{i}=\frac{1}{n\,m_{q}(x_{i})}$. The measure $m_{q}(x_{i})$ is the $q$-deformed form of the previous measure $m(x_{i})$ in Eq. (3) to account for the nonadditivity as also noted in Ref. Abecomment (see Eq. (10) therein). Therefore, the probability normalization condition in Eq. (4) is satisfied in the case of the nonadditive $q$-entropy as well albeit now under $m_{q}(x_{i})$ so that

[TABLE]

Note now that using Eqs. (8) and (9), the following relation is seen to hold

[TABLE]

The substitution of the relation above into Eq. (10) yields the analogous expression of the Shannon entropy in Eq. (6)

[TABLE]

Finally, taking the limit $n\rightarrow\infty$, we obtain the continuous form of the discrete nonadditive entropy as

[TABLE]

where we omitted the divergent term $\lim_{n\to\infty}\ln_{2-q}(n)$ due to the same reason we omitted $\lim_{n\to\infty}\ln(n)$ in the Shannon case in Eq. (6). Namely, the physical observable is not the entropy itself but its change $\Delta S$, so that the divergence $\lim_{n\rightarrow\infty}\ln(n)$ in Eq. (6) and $\lim_{n\rightarrow\infty}\ln_{2-q}(n)$ in Eq. (13) for the Shannon and Tsallis entropy, respectively, vanishes, allowing the entropic structure to converge in the energy continuum.

Another issue worth noting is that the negative of the continuous expression $S\left(\rho\right)$ in Eq. (7) for the BG entropy is nothing but the relative entropy (also known as Kullback-Leibler divergence) Jaynes ; AbeBagci , which reads

[TABLE]

i.e., $-S\left(\rho\right)=K\left[\rho\|m\right]$.

Considering now the negative of the continuous nonadditive $q$-entropy in Eq. (13), we have

[TABLE]

where we have used the relation $-\ln_{q}(x)=\ln_{2-q}(1/x)$ OikBagci2010 . The last expression above is exactly the Csiszár-type nonadditive relative entropy $K_{q}\left[\rho\|m\right]$ (see Eq. (24) in Ref. AbeBagci or Ref. relentfirst for example). In other words, just as its additive counterpart, i.e., $-S\left(\rho\right)=K\left[\rho\|m\right]$, the nonadditive entropy preserves the relation $-S_{q}\left(\rho\right)=K_{q}\left[\rho\|m\right]$ between its continuous generalization and the concomitant relative entropy expression.

So far we have shown that the term $n^{q-1}$ in Eq. (8) is essential, in the discrete case, to correctly obtain the concomitant continuous expression. The presence of this factor can further be elucidated by noting that the discrete entropy is maximized when the states are uniformly distributed. In other words, if we consider the discrete form of the relative entropy expression in Eq. (14) with a uniformly distributed prior, i.e., $r_{i}=1/n$, then one obtains

[TABLE]

where $S(\{p\})$ denotes the discrete BG entropy in Eq. (1). The relation above shows that the entropy maximization is equivalent to the relative entropy minimization when the prior is chosen to be uniform Shore . Therefore, the maximum entropy principle is a particular case of the relative entropy minimization.

A similar calculation using the discrete form of the nonadditive relative entropy $K_{q}\left[p\|r\right]$ in Eq. (15) with a uniform prior yields

[TABLE]

where the first expression on the right-hand side of the equality above is exactly the discrete entropy adopted in Eq. (8). In other words, the minimum relative entropy with a uniform prior is equivalent to the maximum discrete $q$-entropy expression $S_{q}(\{p\})=n^{q-1}\sum_{i=1}^{n}p_{i}\ln_{q}(1/p_{i})$, which explains the discrete form of the $q$-entropy adopted in Eq. (8) Jizba1 ; Jizba2 .

To conclude, we have shown that the discrete nonadditive $q$-entropy does indeed converge for any $q$ values. Moreover, the negative of the continuous $q$-entropy is shown to lead to the (Csiszár-type) $q$-relative entropy mimicking exactly the relation between the negative of the continuous BG expression and the Kullback-Leibler relative entropy. Therefore, there is no obstacle for the use of the $q$-entropy to the continuous classical physical systems as many applications in the field also indicate Tsallisbook .