Note on bounds for symmetric divergence measures

TL;DR

This paper extends existing bounds on symmetric divergence measures by introducing classical q-extensions and non-commutative extensions, building on prior results by Gilardoni and Sason.

Contribution

It provides new extensions of tight bounds for symmetric divergence measures, including q-extensions and non-commutative cases, advancing theoretical understanding.

Findings

Derived classical q-extensions of divergence bounds

Developed non-commutative extensions for divergence measures

Built upon Gilardoni and Sason's foundational results

Abstract

I. Sason obtained the tight bounds for symmetric divergence measures are derived by applying the results established by G. L. Gilardoni. In this article, we are going to report two kinds of extensions for the above results, namely classical q-extension and non-commutative extension.

Equations

$$d_{TV}(P,Q)\equiv \frac{1}{2}\sum _x|P(x)-Q(x)|=\frac{1}{2}||P-Q|{|}_1,$$

$$D_f(P||Q)\equiv \sum _{x}Q(x)f\left(\frac{P(x)}{Q(x)}\right)$$

$$D_q(P||Q)\equiv -\sum _{x}P(x){\ln }_q\frac{Q(x)}{P(x)}=\sum _x\frac{P(x)-P(x{)}^{q}Q(x{)}^{1-q}}{1-q}.$$

$$\underset{P,Q:d_{TV}\left(P,Q\right)=\epsilon }{\inf }{D}_f\left(P\Vert Q\right)=\left(1-\epsilon \right)f\left(\frac{1+\epsilon }{1-\epsilon }\right)-2f^{\prime }\left(1\right)\epsilon$$

$$\overline{C_q}\left(P,Q\right)\equiv D_q\left(P\Vert \frac{P+Q}{2}\right)+D_q\left(Q\Vert \frac{P+Q}{2}\right).$$

$$\underset{P,Q:d_{TV}\left(P,Q\right)=\epsilon }{\min }\overline{C_q}\left(P,Q\right)=-\left(1-\epsilon \right){\ln }_q\frac{1}{1-\epsilon }-\left(1+\epsilon \right){\ln }_q\frac{1}{1+\epsilon },$$

$$J_q\left(P,Q\right)\equiv \frac{1}{2}\left\{D_q\left(P\Vert Q\right)+D_q\left(Q\Vert P\right)\right\}.$$

$$\underset{P,Q:d_{TV}\left(P,Q\right)=\epsilon }{\min }{J}_q\left(P,Q\right)=-\frac{1}{2}\left\{\left(1+\epsilon \right){\ln }_q\frac{1-\epsilon }{1+\epsilon }+\left(1-\epsilon \right){\ln }_q\frac{1+\epsilon }{1-\epsilon }\right\}.$$

$$D_q\left(P\Vert Q\right)\ge \frac{1}{2}{d}_{TV}{\left(P,Q\right)}^{2}forq\ge 1.$$

$$-x{\ln }_q\frac{y}{x}-\left(1-x\right){\ln }_q\frac{1-y}{1-x}\ge -x\log \frac{y}{x}-\left(1-x\right)\log \frac{1-y}{1-x}\ge 2{\left(x-y\right)}^2$$

$$\frac{1}{2}\sum _{u\in \mathcal{U}}\left|p\left(u\right)-Q_{d,l}\left(u\right)\right|\le \min \left\{1,\sqrt{\frac{{\Delta }_{d,q}{\log }_{e}d}{2}}\right\}.$$

$$D_q\left(P\Vert Q_{d,l}\right)\ge D_q\left(\hat{P}\Vert \widehat{Q_{d,l}}\right)\ge 2{\left(P\left(A\right)-Q_{d,l}\left(A\right)\right)}^2=2{\left(\frac{1}{2}{d}_{TV}\left(P,Q_{d,l}\right)\right)}^2=\frac{1}{2}{\left(\sum _{u\in \mathcal{U}}\left|p\left(u\right)-Q_{d,l}\left(u\right)\right|\right)}^2,$$

$$D_q\left(P\Vert Q_{d,l}\right)$$

$${\cal U}=\left(\begin{array}[]{l}\,\,{u_{1}},\,\,\,\,\,{u_{2}},\,\,\,\,\,{u_{3}}\\ 0.5,\,\,\,0.3,\,\,\,0.2\end{array}\right),$$

$$-\frac{1}{{\log }_{e}d}\sum _{u\in \mathcal{U}}p(u{)}^q{\ln }_q{d}^{-l(u)}+\frac{1}{{\log }_{e}d}\sum _{u\in \mathcal{U}}p(u){\log }_e{d}^{-l(u)}-\frac{1}{{\log }_{e}d}\sum _{u\in \mathcal{U}}p(u){\log }_{e}p(u)+\frac{1}{{\log }_{e}d}\sum _{u\in \mathcal{U}}p(u{)}^q{\ln }_{q}p(u)\ge 0.$$

$$d\left(\rho ,\sigma \right)\equiv \frac{1}{2}Tr\left|\rho -\sigma \right|,F\left(\rho ,\sigma \right)\equiv Tr\left|{\rho }^{1/2}{\sigma }^{1/2}\right|,$$

$$1-d\left(\rho ,\sigma \right)\le F\left(\rho ,\sigma \right)\le \sqrt{1-d{\left(\rho ,\sigma \right)}^2}.$$

$$\mathop{\min}\limits_{\rho,\sigma:d\left({\rho,\sigma}\right)=\varepsilon}{C_{Q}}\left({\rho,\sigma}\right)=\left\{\begin{array}[]{l}-\frac{1}{2}\log\left({1-{\varepsilon^{2}}}\right),\varepsilon\in\left[{0,1}\right)\\ \,\,\,+\infty,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\varepsilon=1\end{array}\right.$$

$$D\left(\rho |\sigma \right)\equiv Tr[\rho (\log \rho -\log \sigma )]\ge \frac{1}{2}Tr{\left[\left|\rho -\sigma \right|\right]}^2$$

$$D\left(\rho |\sigma \right)\ge -2\log Tr\left[{\rho }^{1/2}{\sigma }^{1/2}\right]\ge Tr{\left[{\rho }^{1/2}-{\sigma }^{1/2}\right]}^2$$

$$D_f\left(\rho |\sigma \right)\ge D_f\left(\mathcal{E}\left(\rho \right)|\mathcal{E}\left(\sigma \right)\right)$$

$$J\left(\rho |\sigma \right)\ge d\left(\rho ,\sigma \right)\log \left(\frac{1+d\left(\rho ,\sigma \right)}{1-d\left(\rho ,\sigma \right)}\right).$$

$$J\left(\rho |\sigma \right)\ge J\left(P|Q\right)\ge d_{TV}\left(P,Q\right)\log \left(\frac{1+d_{TV}\left(P,Q\right)}{1-d_{TV}\left(P,Q\right)}\right)=d\left(\rho ,\sigma \right)\log \left(\frac{1+d\left(\rho ,\sigma \right)}{1-d\left(\rho ,\sigma \right)}\right).$$

$$\frac{1}{2}\sum _{u\in \mathcal{U}}\left|p\left(u\right)-Q_{d,l,q}\left(u\right)\right|\le \min \left\{1,\sqrt{\frac{{\Delta }_{d,q}{\log }_{e}d}{2}}\right\}$$

$$D_q\left(P\Vert Q_{d,l,q}\right)\ge \frac{1}{2}{\left(\sum _{u\in \mathcal{U}}\left|p\left(u\right)-Q_{d,l,q}\left(u\right)\right|\right)}^2,$$

$$D_q\left(P\Vert Q_{d,l,q}\right)$$

$${\exp_{q}}\left(x\right)=\left\{\begin{array}[]{l}{\left({1+\left({1-q}\right)x}\right)^{\frac{1}{{1-q}}}}\,,{\rm{if}}\,\,1+\left({1-q}\right)x>0\\ \,\,\,\,\,0\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,,{\rm{otherwise}}\end{array}\right.$$

Full text

1 INTRODUCTION

In the paper [1], the tight bounds for symmetric divergence measures are derived by applying the results established in the paper [2]. In the paper [1], the minimization problem for Bhattacharyya coefficient, Chernoff information, Jensen-Shannon divergence and Jeffrey’s divergence under the constraint on total variation distance. In this article, we are going to report two kinds of extensions for the above results, namely classical $q$-extension and non-commutative(quantum) extension. The parametric $q$-extension means that Tsallis entropy $H_{q}(X)\equiv\sum_{x}\frac{p(x)^{q}-p(x)}{1-q}$ [3] converges to Shannon entropy when $q\to 1$. Namely, all results with the parameter $q$ recover the usual (standard) Shannon’s results when $q\to 1$. We give here list of our extensions as follows.

• (i)

The lower bound for Jensen-Shannon-Tsallis diverence is given by applying the results in [2].

• (ii)

The lower bound for Jeffrey-Tsallis divergence is given by applying the results in [2] and deriving $q$-Pinsker’s inequality for $q\geq 1$. This implies new upper bounds of $\sum_{u\in{\cal U}}|p(u)-Q_{d,l}(u)|$.

• (iii)

The lower bound for quantum Chernoff information is given by the known relation between the trace distance and fidelity.

• (iv)

The lower bound for quantum Jeffrey divergence is given by applying the monotonicity (data processing inequality) of quantum $f$-divergence.

2 $q$-EXTENDED CASES

Here we review some quantities. The total variation distance between two probability distributions $P(x)$ and $Q(x)$ is defined by

[TABLE]

where $||\cdot||_{1}$ represents $l_{1}$ norm. The $f$-divergence introduced by Csiszár in [4] is defined by

[TABLE]

where $f$ is convex function and $f(1)=0$. If we take $f(t)=-t\ln_{q}\frac{1}{t}$, where $\ln_{q}(x)\equiv\frac{x^{1-q}-1}{1-q}$ is $q$-logarithmic function defined for $x\geq 0$ and $q\neq 1$, then $f$-divergence is equal to the Tsallis relative entropy (Tsallis divergence) defined by (see e.g., [5])

[TABLE]

In this section, we use the result established by Gilardoni in [2] for the symmetric divergence.

Theorem (Gilardoni, 2006 [2]) We suppose $D_{f}$ is symmetric divergence (which condition is known as $f(u)=uf(1/u)+c(u-1)$, $u\in(0,\infty)$ and $c$ is constant number) and $f:(0,\infty)\to\mathbb{R}$ with $f(1)=0$. Then we have

[TABLE]

As corollaries of the above theorem, we obtain the following two propositions. We define the Jensen-Shannon-Tsallis diverence as

[TABLE]

Then ${D_{{f_{q}}}}\left({P\left\|Q\right.}\right)=\overline{{C_{q}}}\left({P,Q}\right)$ with ${f_{q}}\left(t\right)=-t{\ln_{q}}\frac{{t+1}}{{2t}}-{\ln_{q}}\frac{{t+1}}{2}$, ${f_{q}}$ is convex, with ${f_{q}}\left(1\right)=0$ and $\overline{{C_{q}}}\left({P,Q}\right)=\overline{{C_{q}}}\left({Q,P}\right)$. Thus we have the following proposition which is $q$-parametric extension of Proposition 3 in [1].

Proposition 1

[TABLE]

The equality is archived when $P=\left({\frac{{1-\varepsilon}}{2},\frac{{1+\varepsilon}}{2}}\right),Q=\left({\frac{{1+\varepsilon}}{2},\frac{{1-\varepsilon}}{2}}\right)$.

We also define Jeffrey-Tsallis divergence as

[TABLE]

Then ${D_{{f_{q}}}}\left({P\left\|Q\right.}\right)={J_{q}}\left({P,Q}\right)$ with ${f_{q}}\left(t\right)=\frac{{\left({{t^{q}}-1}\right){{\ln}_{q}}t}}{2}$, ${f_{q}}$ is convex with ${f_{q}}\left(1\right)=0$ and ${J_{q}}\left({P,Q}\right)={J_{q}}\left({Q,P}\right)$. Thus we have the following proposition which is $q$-parametric extension of Proposition 4 in [1].

Proposition 2

[TABLE]

The equality is archived when $P=\left({\frac{{1-\varepsilon}}{2},\frac{{1+\varepsilon}}{2}}\right),Q=\left({\frac{{1+\varepsilon}}{2},\frac{{1-\varepsilon}}{2}}\right)$.

Here we are able to prove the following lemma, which may be named $q$-Pinsker’s inequality.

Lemma 1

[TABLE]

Proof: The proof is easily done by the fact that $\log t\leq\frac{{{t^{r}}-1}}{r},\left({t>0,r>0}\right)$ implies $-\log\frac{1}{t}\leq-{\ln_{q}}\frac{1}{t},\left({t>0,q>1}\right)$, putting $r=q-1$. Thus we have

[TABLE]

for ${0<x,y<1,q\geq 1}$. Thus we have this lemma by data processing inequality.

As remark, the above $q$-Pinsker inequality does not hold for the case $0<q<1$, since we have counter-examples. Applying this lemma, we can prove the following proposition, which condition is same to the paper [1] except for the extended parameter $q$.

Theorem 1 Consider a memoryless stationary source with alphabet ${\cal U}$ with probability distribution $P$ and assume that a uniquely decodable code with an alphabet size $d$. For $q\geq 1$, we have

[TABLE]

Where ${\Delta_{d,q}}\equiv{\overline{n}_{q}}-{H_{d,q}}\left({\cal U}\right)$,${\overline{n}_{q}}\equiv-\frac{\left(c_{d,l}\right)^{q-1}}{\log_{e}d}\sum\limits_{u\in{\cal U}}{p{{\left(u\right)}^{q}}\ln_{q}d^{-l\left(u\right)}}$,${H_{d,q}}\left({\cal U}\right)\equiv-\frac{1}{{{{\log}_{e}}d}}\sum\limits_{u\in{\cal U}}{p{{\left(u\right)}^{q}}{{\ln}_{q}}p\left(u\right)}$, ${Q_{d,l}}\left(u\right)\equiv\frac{d^{-l(u)}}{{{c_{d,l}}}}$ and ${c_{d,l}}\equiv\sum\limits_{u\in{\cal U}}d^{-l(u)}.$

Proof: We give the sketch of the proof of this proposition. Firstly $\sum\limits_{u\in{\cal U}}{\left|{p\left(u\right)-{Q_{d,l}}\left(u\right)}\right|}\leq 2$ is trivial. By Lemma 1, we have

[TABLE]

where $A\equiv\left\{{x:P\left(x\right)>{Q_{d,l}}\left(x\right)}\right\}$, $Y\equiv\phi\left(X\right)$ and $\widehat{P}$ and $\widehat{{Q_{d,l}}}$ are distributions of new random variable $Y$. By simple computations with formula $\ln_{q}\frac{y}{x}=x^{q-1}(\ln_{q}y-\ln_{q}x)$, we have

[TABLE]

since the Kraft-McMillian inequality $c_{d,l}\leq 1$ was used. Thus we have $\frac{1}{2}\left(\sum\limits_{u\in{\cal U}}{\left|{p\left(u\right)-{Q_{d,l}}\left(u\right)}\right|}\right)^{2}\leq{\log_{e}}d\cdot\,{\Delta_{d,q}}.$

Remark 1 This theorem is a parametric extension of the inequality (32) in the paper [1] in the sense that the left hand side of our inequality contains the parameter $q\geq 1$. We also note that the condition $q\geq 1$ is corresponding to the result in our previous paper [6], so the condition $q\geq 1$ may not be so unnatural within our framework of this topic.

In addition, we compare our upper bound with parameter $q\geq 1$ obtained in Theorem 1 and that obtained in the paper [1]. Actually we give an example such that $\sqrt{\frac{\Delta_{d,q}\log_{e}d}{2}}\leq\sqrt{\frac{\Delta_{d,1}\log_{e}d}{2}}$, where $\Delta_{d,1}$ was used in the paper [1] as $\Delta_{d}$. Consider the following information source

[TABLE]

with $d=2$. Then we have the code $u_{1}\to``0",u_{2}\to``10",u_{3}\to``110"$ by Shannon-Fano coding, so that $c_{d,l}=\frac{7}{8}<1$ since $l_{1}=1,l_{2}=2,l_{3}=3$. By numerical computations, we have $\sqrt{\frac{\Delta_{2,1.5}\log_{e}2}{2}}\simeq 0.225793$ and $\sqrt{\frac{\Delta_{2,1}\log_{e}2}{2}}\simeq 0.272669$. This means there exists a code such that $\sqrt{\frac{\Delta_{d,q}\log_{e}d}{2}}\leq\sqrt{\frac{\Delta_{d,1}\log_{e}d}{2}}$, which shows our upper bound with the parameter $q\geq 1$ is tighter than the upper bound in the paper [1], in this example. We performed some numerical computations with a few information sources, then we could find the parameter $q\geq 1$ such that $\sqrt{\frac{\Delta_{d,q}\log_{e}d}{2}}\leq\sqrt{\frac{\Delta_{d,1}\log_{e}d}{2}}$ for the case $c_{d,l}<1$.

However, for the case $c_{d,l}=1$ (e.g., Huffman code), the following proposition can be proven.

Proposition 3 Let $q\geq 1$ and $c_{d,l}=1$. Then we have the relation $\Delta_{d,1}\leq\Delta_{d,q}$.

Proof: We firstly prove the inequality $f_{q}(x,y)\geq 0$ for $q\geq 1,0<x,y\leq 1$, where $f_{q}(x,y)\equiv x(\log_{e}y-\log_{e}x)+x^{q}(\ln_{q}x-\ln_{q}y).$ Since $\frac{df_{q}(x,y)}{dy}=\frac{x^{q}}{y^{q}}\left(\frac{x^{1-q}}{y^{1-q}}-1\right)$, if $x\leq y$, then $\frac{df_{q}(x,y)}{dy}\geq 0$ and if $x\geq y$, then $\frac{df_{q}(x,y)}{dy}\leq 0$, thus we have $f_{q}(x,y)\geq f_{q}(x,x)=0$. Putting $x=p(u)$ and $y=d^{-l(u)}$, taking summation on both sides by $u\in{\cal U}$ and dividing the both sides by $\log_{e}d$, we have

[TABLE]

When $c_{d,l}=1$, we thus obtain the inequality $\Delta_{d,q}-\Delta_{d,1}={\overline{n}_{q}}-{\overline{n}_{1}}+H_{d,1}({\cal U})-H_{d,q}({\cal U})\geq 0$, taking account that the usual average code length can be rewritten as ${\overline{n}_{1}}=\sum_{u\in{\cal U}}p(u)l(u)=-\frac{1}{\log_{e}d}\sum_{u\in{\cal U}}p(u)\log_{e}d^{-l(u)}$.

This proposition shows that for the special (but nontrivial) case $c_{d,l}=1$, the upper bound $\sqrt{\frac{\Delta_{d,1}\log_{e}d}{2}}$ given in (32) of the paper [1] is always tighter than ours $\sqrt{\frac{\Delta_{d,q}\log_{e}d}{2}}$ (for $q\geq 1$) obtained in Theorem 1.

3 NON-COMMUTATIVE CASES

Let $\rho$ and $\sigma$ be density matrices (quantum states), which are positive semi-definite matrices and unit trace. Then the following quantities are well known in the field of quantum information or physics as trace distance and fidelity, respectively:

[TABLE]

Where $|A|=(A^{*}A)^{1/2}$. Then we have the following propositions.

Proposition 4 For the trace distance and fidelity, we have the following relation:

[TABLE]

This relation is well known in the field of quantum information or quantum statistical physics, and this proposition is non-commutative extension of Proposition 1 in the paper [1].

By the easy calculations such as ${C_{Q}}\left({\rho,\sigma}\right)\equiv-\log\left({\mathop{\min}\limits_{0\leq s\leq 1}Tr\left[{{\rho^{s}}{\sigma^{1-s}}}\right]}\right)=-\mathop{\min}\limits_{0\leq s\leq 1}\left({\log Tr\left[{{\rho^{s}}{\sigma^{1-s}}}\right]}\right)\,\geq-\log Tr\left[{{\rho^{1/2}}{\sigma^{1/2}}}\right]\geq-\log Tr\left[{\left|{{\rho^{1/2}}{\sigma^{1/2}}}\right|}\right]=-\log F\left({\rho,\sigma}\right)\geq-\frac{1}{2}\log\left({1-d{{\left({\rho,\sigma}\right)}^{2}}}\right)$, we have the following proposition.

Proposition 5 For the quantum Chernoff information, we have

[TABLE]

The above proposition is also non-commutative extension of Proposition 2 in the paper [1].

The quantum Pinsker inequality on quantum relative entropy (divergence) and similar one are known (see e.g., [7] and [8], respectively)

[TABLE]

and

[TABLE]

To show our final result, we use the following well-known fact. See [7] for example.

Lemma 2 Let ${\cal E}:B({\cal H})\to B({\cal K})$ be a state transformation. For an operator monotone decreasing function $f:\mathbb{R}^{+}\to\mathbb{R}$, the monotonicity holds:

[TABLE]

where ${D_{f}}\left({\rho\left|\sigma\right.}\right)\equiv Tr\left[{\rho f\left(\Delta\right)\left(I\right)}\right]$ is the quantum $f$-divergence, with ${\Delta_{\sigma,\rho}}\equiv\Delta=LR$ is the relative modular operator such as $L\left(A\right)=\sigma A$ and $R\left(A\right)=A{\rho^{-1}}$.

Theorem 2 The quantum Jeffrey divergence defined by $J\left({\rho\left|\sigma\right.}\right)\equiv\frac{1}{2}\left\{{D\left({\rho\left|\sigma\right.}\right)+D\left({\sigma\left|\rho\right.}\right)}\right\}$ has the following lower bound:

[TABLE]

Proof: By Lemma 2, Proposition 4 in the paper [1] and ${\left\|{\rho-\sigma}\right\|_{1}}={\left\|{P-Q}\right\|_{1}}$(which will be shown in the end of proof), we have

[TABLE]

Here we note that $f\left(t\right)=\frac{1}{2}\left({t-1}\right)\log t$ is operator convex which is equivalent to operator monotone decreasing and we have ${D_{\frac{1}{2}\left({t-1}\right)\log t}}\left({\rho\left|\sigma\right.}\right)=J\left({\rho\left|\sigma\right.}\right)$, since $\left({{\Delta_{\sigma,\rho}}\log{\Delta_{\sigma,\rho}}}\right)\left(Y\right)=\sigma\log\sigma\left(Y\right){\rho^{-1}}-\sigma{\rho^{-1}}\log\rho\left(Y\right)$.

Finally, we show ${\left\|{\rho-\sigma}\right\|_{1}}={\left\|{P-Q}\right\|_{1}}$. Let ${\cal A}=C^{*}(\rho_{1}-\rho_{2})$ be commutative $C^{*}$-algebra generated by $\rho_{1}-\rho_{2}$, $M_{n}$ be the set of all $n\times n$ matrices and set the map ${\cal E}:M_{n}\to{\cal A}$ as trace preserving, conditional expectation. If we take $p_{1}={\cal E}(\rho_{1})$ and $p_{2}={\cal E}(\rho_{2})$, then two elements ${\left({{\rho_{1}}-{\rho_{2}}}\right)_{+}}$ and ${\left({{\rho_{1}}-{\rho_{2}}}\right)_{-}}$ of Jordan decomposition of $\rho_{1}-\rho_{2}$, are commutative functional calculus of $\rho_{1}-\rho_{2}$, and we have ${p_{1}}-{p_{2}}={\cal E}\left({{\rho_{1}}-{\rho_{2}}}\right)={\cal E}\left({{{\left({{\rho_{1}}-{\rho_{2}}}\right)}_{+}}-{{\left({{\rho_{1}}-{\rho_{2}}}\right)}_{-}}}\right)\,={\cal E}\left({{{\left({{\rho_{1}}-{\rho_{2}}}\right)}_{+}}}\right)-{\cal E}\left({{{\left({{\rho_{1}}-{\rho_{2}}}\right)}_{-}}}\right)={\left({{\rho_{1}}-{\rho_{2}}}\right)_{+}}-{\left({{\rho_{1}}-{\rho_{2}}}\right)_{-}}\,={\rho_{1}}-{\rho_{2}}$ which implies ${\left\|{\rho-\sigma}\right\|_{1}}={\left\|{P-Q}\right\|_{1}}$.

4 ACKNOWLEDGMENTS

The author (S. F.) was partially supported by JSPS KAKENHI Grant Number 16K05257.

5 Appendix: Added notes related to Theorem 1

Actually we have $\lim_{q\to 1}{\overline{n}_{q}}=\sum_{u\in{\cal U}}p(u)l(u)$ which is the usual average code length, but the definition of ${\overline{n}_{q}}$ in Theorem 1 seems to be complicated and somewhat unnatural to understand its meaning. In order to overcome this problem, we may adopt the simple alternative definition for ${\overline{n}_{q}}$ instead of that in Theorem 1. Then we have the following proposition.

Proposition A Let $q\geq 1$ and $c_{d,l,q}\leq 1$. Then we have

[TABLE]

Where ${\Delta_{d,q}}\equiv{\overline{n}_{q}}-{H_{d,q}}\left({\cal U}\right)$,${\overline{n}_{q}}\equiv\sum\limits_{u\in{\cal U}}{p{{\left(u\right)}^{q}}l\left(u\right)}$,${H_{d,q}}\left({\cal U}\right)\equiv-\frac{1}{{{{\log}_{e}}d}}\sum\limits_{u\in{\cal U}}{p{{\left(u\right)}^{q}}{{\ln}_{q}}p\left(u\right)}$, ${Q_{d,l,q}}\left(u\right)\equiv\frac{1}{{{c_{d,l,q}}}}{\exp_{q}}\left({{{\log}_{e}}{d^{-l\left(u\right)}}}\right)$ and ${c_{d,l,q}}\equiv\sum\limits_{u\in{\cal U}}{{{\exp}_{q}}\left({{{\log}_{e}}{d^{-l\left(u\right)}}}\right)}$, where $q$-exponential function $\exp_{q}(\cdot)$ is the inverse function of $q$-logarithmic function $\ln_{q}(\cdot)$ and its form is given in the proof of this proposition.

Proof: By the same way to the proof of Theorem 1, we have

[TABLE]

By simple computations with formula $\ln_{q}\frac{y}{x}=y^{1-q}(\ln_{q}y+\ln_{q}\frac{1}{x})$, we have

[TABLE]

since $d\geq 2$, $l(u)\geq 1$ implies $\log_{e}d^{-l(u)}\leq 0$ thus we have $1+(1-q)\log_{e}d^{-l(u)}\geq 0$, then the definition of $q$-exponential function

[TABLE]

shows $\exp_{q}(\log_{e}d^{-l(u)})\geq 0$ and $c_{d,l,q}\leq 1$ was used. Thus we have $\frac{1}{2}\left(\sum\limits_{u\in{\cal U}}{\left|{p\left(u\right)-{Q_{d,l,q}}\left(u\right)}\right|}\right)^{2}\leq{\Delta_{d,q}{\log_{e}}d}.$

We could not remove the needless and meaningless condition $c_{d,l,q}\leq 1$ in the above proposition, unfortunately. It is known that the inequality $c_{d,l,1}\leq 1$ holds for the uniquely decodable code and the equality $c_{d,l,1}=1$ holds if the code archives the entropy, namely ${\overline{n}_{1}}=H_{d,1}({\cal U})$ [1]. In our proposition, we obtained $q$-parametric extension but it does not have any information theoretical meaning. We will have to consider about this problem in the future.