Uniqueness and characterization theorems for generalized entropies

TL;DR

This paper proves that, under mild conditions, the Tsallis entropy is the only composable generalized entropy in trace form, and characterizes a broad class of non-trace-form entropies like Rényi's.

Contribution

It establishes the uniqueness of Tsallis entropy among trace-form generalized entropies and characterizes non-trace-form entropies with similar composability properties.

Findings

Tsallis entropy is the only trace-form composable generalized entropy.

Characterization of a broad class of non-trace-form entropies like Rényi's.

Implications for studying complex systems with generalized entropies.

Abstract

The requirement that an entropy function be composable is key: it means that the entropy of a compound system can be calculated in terms of the entropy of its independent components. We prove that, under mild regularity assumptions, the only composable generalized entropy in trace form is the Tsallis one-parameter family (which contains Boltzmann-Gibbs as a particular case). This result leads to the use of generalized entropies that are not of trace form, such as R\'enyi's entropy, in the study of complex systems. In this direction, we also present a characterization theorem for a large class of composable non-trace-form entropy functions with features akin to those of R\'enyi's entropy.

Equations

$$S(A\cup B)=\Phi (S(A),S(B))$$

$$\Phi (x,y)=\Phi (y,x)\Phi (x,0)=x$$

$$\Phi (x,\Phi (y,z))=\Phi (\Phi (x,y),z)$$

$$p=(p_1,\dots ,p_W),p_i⩾0,\sum _{i=1}^W{p}_i=1.$$

$$S_{\mathrm{BG}}(p):=\sum _{i=1}^W{p}_i\ln \frac{1}{p_i}.$$

$$S(p)=\sum _{i=1}^{W}f(p_i),$$

$$f(0)=f(1)=0.$$

$$S_q(p):=\sum _{i=1}^W{f}_q(p_i),f_q(t):=\frac{t-t^q}{q-1}.$$

$$f(t)=c{f}_q(t),$$

$$S_{\mathrm{R}}(p):=\frac{1}{1-\alpha}\ln\bigg{(}\sum_{i=1}^{W}p_{i}^{\alpha}\bigg{)},\qquad\alpha>0$$

$$\widetilde{S}(p)=g\Bigg{(}\sum_{i=1}^{W}h(p_{i})\Bigg{)}\,,$$

$$h(0)=g(h(1))=0.$$

$$h(t)=at+b{t}^q$$

$$p^A=(p_i^A{)}_{i=1}^W\text{and}{p}^B=(p_j^B{)}_{j=1}^{W^{\prime }},$$

$$p^{A\cup B}=(p_{ij}^{A\cup B}{)}_{1⩽i⩽W,1⩽j⩽W^{\prime }},$$

$$p_{ij}^{A\cup B}:=p_i^A{p}_j^B.$$

$$\sum _{i=1}^W\sum _{j=1}^{W^{\prime }}F(p_i^A{p}_j^B).$$

$$\sum _{i=1}^W\sum _{j=1}^{W^{\prime }}F(p_i^A{p}_j^B),\sum _{i=1}^{W}F(p_i^A)\text{and}\sum _{j=1}^{W^{\prime }}F(p_j^B)$$

$$\sum_{i=1}^{W}\sum_{j=1}^{W^{\prime}}F(p_{i}^{A}p_{j}^{B})=\Psi\bigg{(}\sum_{i=1}^{W}F(p_{i}^{A}),\sum_{j=1}^{W^{\prime}}F(p_{j}^{B})\bigg{)}\,.$$

$$\Psi (x,y)=a_0+a_{1}x+a_{2}y+a_{3}xy$$

$${\mathcal{P}}_W$$

$$p(s)=\bigg{(}p^{A}_{1}+sP_{1},p^{A}_{2}+sP_{2},\dots,p^{A}_{W-1}+sP_{W-1},p^{A}_{W-1}-s\sum_{l=1}^{W-1}P_{l}\bigg{)}$$

$$P:=(P_1,\dots ,P_{W-1})$$

$$\sum_{i=1}^{W}\sum_{j=1}^{W^{\prime}}F(p_{i}(s)\,p_{j}^{B})-\Psi\bigg{(}\sum_{i=1}^{W}F(p_{i}(s)),\sum_{j=1}^{W^{\prime}}F(p_{j}^{B})\bigg{)}=0$$

$$\sum_{l=1}^{W-1}\bigg{(}\sum_{j=1}^{W^{\prime}}p_{j}^{B}\,\big{[}F^{\prime}(p_{l}^{A}p_{j}^{B})-F^{\prime}(p_{W}^{A}p_{j}^{B})\big{]}\\ -D_{1}\Psi\bigg{(}\sum_{i=1}^{W}F(p_{i}^{A}),\sum_{j=1}^{W^{\prime}}F(p_{j}^{B})\bigg{)}\,\big{[}F^{\prime}(p_{l}^{A})-F^{\prime}(p_{W}^{A})\big{]}\bigg{)}\,P_{l}=0\,,$$

$$\sum_{j=1}^{W^{\prime}}p_{j}^{B}\,\big{[}F^{\prime}(p_{l}^{A}p_{j}^{B})-F^{\prime}(p_{W}^{A}p_{j}^{B})\big{]}\\ =D_{1}\Psi\bigg{(}\sum_{i=1}^{W}F(p_{i}^{A}),\sum_{j=1}^{W^{\prime}}F(p_{j}^{B})\bigg{)}\,\big{[}F^{\prime}(p_{l}^{A})-F^{\prime}(p_{W}^{A})\big{]}$$

$$D_{1}\Psi\bigg{(}\sum_{i=1}^{W}F(p_{i}^{A}),\sum_{j=1}^{W^{\prime}}F(p_{j}^{B})\bigg{)}=a\bigg{(}\sum_{j=1}^{W^{\prime}}F(p_{j}^{B})\bigg{)}$$

$$\frac{\partial }{\partial x}\Psi (x,y)=a(y).$$

$$\Psi (x,y)=a(y)x+b(y),$$

$$\frac{\partial }{\partial y}\Psi (x,y)=\tilde{a}(x)$$

$$\Psi (x,y)=\tilde{a}(x)y+\tilde{b}(x),$$

$$\Psi (x,y)=a_0+a_{1}x+a_{2}y+a_{3}xy$$

$$\sum_{i=1}^{W}\sum_{j=1}^{W^{\prime}}f(p_{i}^{A}p_{j}^{B})=\Phi\bigg{(}\sum_{i=1}^{W}f(p_{i}^{A}),\sum_{j=1}^{W^{\prime}}f(p_{j}^{B})\bigg{)}\,.$$

Full text

This result leads to the use of generalized entropies that are not of trace form, such as Rényi’s entropy, in the study of complex systems. In this direction, we also present a characterization theorem for a large class of composable non-trace-form entropy functions with features akin to those of Rényi’s entropy.

1. 1 Introduction
2. 2 A lemma on the functional dependence of traces
3. 3 Proof of Theorem 1
4. 4 Proof of Theorem 2
5. 5 The cases of absolutely continuous or analytic functions

1. Introduction

The first example of a generalized entropy in information theory goes back to the pioneering work of Rényi [17] in the early ’60s, where he introduced a one-parameter family of entropies that reduces to the classical Shannon entropy for a concrete value of the parameter. Rényi was interested in the form of the most general information measure satisfying certain natural requirements, in particular additivity with respect to the composition of independent statistical systems. The entropy introduced by Tsallis in 1988 in [22] has been widely investigated too: it is the non-additive entropic form most widely used in the natural and social sciences (see [27] for an updated bibliography on the issue).

After these works, many different generalized entropic functions have been constructed as non-additive information measures of a statistical system (see, e.g., [4], [12], [1], [16], [10]). The main motivation for these new entropic forms lies, generally speaking, in the emergence of the theory of complex systems, which often exhibits new phenomena that require novel carefully designed information-theoretical tools for their interpretation. For instance, several entropies other than Boltzmann–Gibbs have played a relevant role in the study of quantum entanglement [6, 7, 8], in the theory of divergences generalizing the classical Kullback-Leibler one [14, 9], in geometric information theory [3], and in theoretical linguistics and social sciences.

Several approaches have been proposed to classify the plethora of entropy functions that have appeared in the literature over the last decades. The standard axiomatic approach is based on the work by Shannon [20] and Khinchin [13], who characterized the Boltzmann entropy (within the class of trace-class entropies, which we will define later) in terms of four requirements, now called the Shannon–Khinchin (SK) axioms. Essentially, these axioms correspond to the hypotheses that entropy, as a function defined on a certain space of probability distributions, be continuous (SK1), expansible (i.e., adding an event of zero probability does not change the entropy) (SK2), and that the uniform distribution maximizes the entropy (SK3). The fourth axiom (SK4) is the additivity of the entropy, that is, that the entropy of the composition of two subsystems is the sum of their individual entropies.

It stands to reason that if one relaxes the additivity condition in axiom (SK4), new possible functional forms of the entropy may arise. A convenient way of doing this is to replace the axiom (SK4) by the weaker composability axiom, introduced in [24]. Roughly speaking, this axiom asserts that the entropy $S$ of a compound system $A\cup B$ consisting of two independent systems $A$ and $B$ should be computable just in terms of the individual entropies of $A$ and $B$. Using the notation $S(A)$ to represent the entropy of the system $A$, this means that there is a function of two variables, $\Phi(x,y)$, such that

[TABLE]

for any independent systems $A$ and $B$. Property (1) is of fundamental importance; indeed, as in the case of the Boltzmann–Gibbs entropy, it implies that an entropic function is properly defined on macroscopic states of a given system, so that it can be computed without knowing any information on the underlying microscopical dynamics. This is the reason for which this property is key to ensure that the entropy is physically meaningful. It should be mentioned that the composability of an entropy also has tangible consequences from an information-theoretical point of view [21].

Let us also recall the classical result by Lieb and Yngvason [15], where the existence of an entropy function is derived from monotonicity, additivity and extensivity requirements for all allowed states. In the proof, the fundamental thermodynamic meaning of the composability of classical entropy is laid bare.

A priori, the requirement that an entropy by composable is actually even stronger than it looks. What we mean by this is that one must take into account that the combination of statistical systems (which is customarily represented as $A\cup B$ but is typically given by a tensor product of vector spaces) is associative and commutative. In addition, if we compound a system with another one in a zero-entropy configuration, the entropy of the compound system should be equal to the entropy of the first system. Therefore, it is crucial to demand that

[TABLE]

When these properties are satisfied, $S$ will be said to be a (strictly) composable generalized entropy [24, 25]. If the composability axiom is satisfied only when the subsystems are described by the uniform distribution, then we shall say that the entropy $S$ is weakly composable. A vast majority of the most popular generalized entropies in the literature are at least weakly composable.

The notion of group entropy is a direct consequence of the previous discussion. In the usual statistical picture, we shall describe a system possessing a certain number $W$ of microstates by a probability density, which is a $W$-component vector of nonnegative reals satisfying

[TABLE]

Hence $p_{i}$ is the probability that the system $A$ is in the $i^{\text{th}}$ microstate. A group entropy is a nonnegative function $S(p_{1},\ldots,p_{W})$ that satisfies the axioms (SK1)–(SK3) and is strictly composable. In other words, in addition to the usual requirements of continuity, expansibility and concavity, a group entropy (due to the properties (1)-(1)) also possesses a group-theoretical structure represented by the product $\Phi(x,y)$ and which holds in all the probability distribution space associated with a given complex system.

Prime examples of group entropies are the Boltzmann and Rényi entropies, which satisfy (1) with the additive law $\Phi(x,y)=x+y$. Non-additive laws have been studied extensively too, since they arise naturally in the context of complex systems, where the entropy of the total system is expected to different from the sum of the entropies of the independent parts. Tsallis’s entropy, for instance, obeys the law $\Phi(x,y)=x+y+(1-q)xy$, corresponding to the multiplicative formal group law. Although we will not rely on these techniques for this paper, it worth mentioning that formal group theory [11], intensively investigated in algebraic topology since the second half of XX century, provides a natural classification of group laws in terms of formal power series, as it offers very general algebraic results and a natural language to formulate the theory of generalized entropies.

The main problem that we address in this paper is to ascertain which is the most general form that a strictly composable generalized entropy can take. For the sake of generality, in the subsequent analysis we will not even assume that $\Phi(x,y)$ be a group law: we shall only assume that it is an arbitrary two-variable function, with suitable regularity properties, and derive from first principles that, in particular, it has to be a group law.

Before stating our results, let us recall the expression of the Boltzmann–Gibbs entropy of a system in a state described by a probability distribution $p=(p_{1},\dots,p_{W})$ reads

[TABLE]

(Throughout this work, we will set Boltzmann’s constant to one: $k_{\mathrm{B}}=1$.) Inspired by this expression, perhaps the most common way of constructing entropy functions is in trace form, which in the above notation means that there is a one-variable nonnegative function $f(t)$ such that

[TABLE]

This condition ensures that the entropy is zero if the probability distribution of a system is $p_{i}=\delta_{i,1}$ (that is, the system is in a certainty state). If a quantum system is described by a density matrix $\rho$, then the corresponding quantum entropy can be directly computed as $\operatorname{tr}f(\rho)$.

Since Shannon’s foundational paper in information theory [20], an extensive body of literature on trace-form entropies has appeared. The prototype example of trace-form generalized entropy is the one-parameter generalization of Boltzmann’s entropy introduced by Tsallis, given for $q\geqslant 0$ and $q\neq 1$ by the formula

[TABLE]

As $q$ tends to $1$ one recovers the Boltzmann–Gibbs entropy, so it is customary to set $S_{1}(p):=S_{\mathrm{BG}}(p)$ and $f_{1}(t):=t\ln\frac{1}{t}$. A two-parameter presentation of the Tsallis entropy was recently introduced in [25].

Our first result shows that, in a way, the class of trace-form entropies has a serious drawback: we prove that, under mild regularity assumptions, the only composable trace-form entropy is the Tsallis entropy, with Boltzmann-Gibbs as a particular case. Consequently, in order to construct new entropies one must either assume that they are not in trace form or to deal with the fact that if one has two independent systems $A$ and $B$, the entropy of the total system will not be determined, in general, in terms of the entropies of the independent subsystems. Of course, this also implies that no group-theoretical structure is available. The result can be stated as follows.

Theorem 1**.**

Let $S$ be an entropy of the form (3) with a function $f$ of class $C^{2}((0,1))\cap C^{1}([0,1])$. Suppose that these entropies satisfy the condition (1) with a composition law $\Phi$ of class $C^{1}$. Then there are positive real constants $c,q$ such that

[TABLE]

so $S$ is the Tsallis entropy (4) for some real $q$, up to a multiplicative constant. The composition law is $\Phi(x,y)=x+y+\alpha xy$ for some explicit constant $\alpha$ depending on $c$ and $q$.

At the same time, it is worth stressing that not all the entropies commonly employed in information theory are of trace form. For example, the celebrated Rényi entropy [17]

[TABLE]

is indeed not in this class. Hence, inspired by the form of Rényi’s entropy, it is natural to consider generalized entropies of the form

[TABLE]

These conditions once again ensure that the entropy of a system with probability distribution $p_{i}=\delta_{i1}$ is zero. The function $g$ is typically assumed to be monotone. Note that the class (5), under the additional constraint that $h$ be a concave function, was first considered in [18]. A quantum version of this family of entropies was studied in [5]. Nontrivial examples of entropy functions of this form are provided by the large class of $Z$-entropies, recently introduced in [25]. They are multiparametric non-trace-form group entropies of the form (5) which, under mild assumptions, reduce to the standard Rényi entropy in a suitable limit.

Our second result is a characterization of the generalized entropies of the form (5) that are composable. Again for the sake of generality, we do not assume that the function $g$ is convex. Our result essentially asserts that, for any given monotone function $g$, a function of the form (5) satisfies the composability condition if and only if the “trace part” is of the form $h(t)=at+bt^{q}$ for some real constants. More precisely, we have the following statement:

Theorem 2**.**

Let $S$ be an entropy of the form (5), where the function $h$ is of class $C^{2}((0,1))\cap C^{1}([0,1])$ and $g$ is a $C^{1}$ function with $g^{\prime}\neq 0$. Then the entropy $S$ satisfies the composability condition (1) with $\Phi$ of class $C^{1}$ if and only if

[TABLE]

for some real constants $a,b$ and $q>1$. The composition law can be written down explicitly in terms of these constants and the function $g$.

It should be remarked that the regularity hypothesis is probably not sharp, but it is crucially used in a rather tricky derivation of differential equations from some (rather unmanageable) functional equations that lies at the core of the proof of the theorems. Notice that the functions $f(t)=(t-t^{q})/(q-1)$ and $h(t)=t^{\alpha}$ that respectively appear in the Tsallis and Rényi entropies are of class $C^{1}$ at 0 precisely for $q>1$ and $\alpha\geqslant 1$, and that we are not obtaining the function $h(t)=at+bt\ln\frac{1}{t}$ because we require $h$ to be continuously differentiable at [math].

Let us conclude the Introduction with some comments about the proof of these results and the organization of the paper. The proof of both theorems, respectively presented in Sections 3 and 4, are based on similar arguments, so let us illustrate them in the case of Theorem 1. The proof of this result involves three ideas. Firstly, an easy argument shows that the function $\Phi$ appearing the composition law must be an associative, commutative product. With some more work, which involves taking variations in the equation with respect to the probability densities, we show that in fact the only admissible composition function is $\Phi(x,y)=x+y+\alpha xy$ for some real constant $\alpha$. A key tool to prove this will be a lemma on the functional independence of certain functions that we present in Section 2. It is worth mentioning that the result on the structure of the composition function holds with less stringent regularity hypotheses (specifically, for any $f$ absolutely continuous on $[0,1]$). In passing from the expression for $\Phi$ to the general form of $f$ we must indeed use the $C^{1}$ regularity of $f$ up to the endpoints of the interval to get rid of some of the several parameters that our argument relies on. Some further observations in this direction are presented in Section 5.

2. A lemma on the functional dependence of traces

In this section we will prove a key lemma on the structure of the functions that satisfy a certain kind of functional relations which is strongly related with the composability condition (1).

To motivate this result, it is convenient to start by recalling a basic definition about independent systems that will be used in the rest of the paper. Given two probability distributions

[TABLE]

It is this formula, together with the expressions (3) and (5) for the entropies that we will study in this paper, which leads us next to consider expressions of the form

[TABLE]

The following lemma, which is the main result of this section, asserts that if the functions

[TABLE]

are functionally dependent (as functions of the probability densities $p^{A}$ and $p^{B}$), then the expression of the first of these functions in terms of the other two is given by a very simple algebraic relation that only depends on four parameters.

Lemma 3**.**

Suppose that there is a one-variable function $F\in C^{1}((0,1))$ and a $C^{1}$ function $\Psi$ of two variables such that, for any probability densities $p^{A}$ and $p^{B}$ as above, one has

[TABLE]

Then necessarily

[TABLE]

for some real constant $a_{j}$.

Proof.

We shall next take variations in the identity (7) with respect to the probability density $p^{A}$. Notice that the space of probability densities with $W$ states is

[TABLE]

which one can understand as a bounded subset of $\mathbb{R}^{W-1}$ with nonempty interior. To consider variations of the probability density $p^{A}$, let us choose without loss of generality a probability density $p^{A}$ that does not lie on the boundary of the set (8), i.e., such that $p^{A}_{i}>0$ for all $i$. Then the curve

[TABLE]

is contained in the set (8) for an arbitrary vector

[TABLE]

in $\mathbb{R}^{W-1}$ with $|P|\leqslant 1$, and for all $s$ in a small enough interval $s\in(-\varepsilon,\varepsilon)$.

Inserting this curve in the identity (7) we obtain

[TABLE]

for all $s\in(-\varepsilon,\varepsilon)$ and all $P$ in the unit ball of $\mathbb{R}^{W-1}$. Differentiating this relation at $s=0$ one then obtains

[TABLE]

where $D_{1}\Psi$ denotes the derivative of the function $\Psi$ with respect to first argument. Given that this relation must hold for all $P\in\mathbb{R}^{W-1}$ with $|P|\leqslant 1$, we infer that for all $1\leqslant l\leqslant W-1$ one has

[TABLE]

As the left hand side of the identity (12) does not depend on $p_{i}^{A}$ for $i\neq l,W$, choosing without loss of generality $W\geqslant 4$ (so that the dimension of the space of probability densities is at least 3) it follows that so must be the right hand side. Hence we infer that

[TABLE]

for some function $a$ or, to put it differently,

[TABLE]

This can be immediately integrated to yield

[TABLE]

with $b$ another arbitrary function.

One can reserve the role of $p^{A}$ and $p^{B}$ and consider variations of the identity (7) with respect to the probability density $p^{B}$. Arguing as above we then infer that

[TABLE]

for some function $\widetilde{a}$, or equivalently

[TABLE]

with $\widetilde{b}$ another arbitrary function. From (28) and (29) we obtain that $\Psi(x,y)$ is a polynomial of order 1 both in $x$ and $y$ (separately), so

[TABLE]

for some real constants $a_{j}$. ∎

3. Proof of Theorem 1

In this section we present the proof of the theorem, which consists of two steps.

Step 1: The composition function is $\Phi(x,y)=x+y+\alpha xy$

Using the notation (6) introduced in the previous section, the composability condition (1) reads as

[TABLE]

Since this relation must hold for all probability distributions $p^{A}$ and $p^{B}$, Lemma 3 ensures that the composition law must be of the form

[TABLE]

for some real constants $a_{j}$.

Let us now evaluate (16) when the second probability distribution is $p^{B}_{j}=\delta_{j1}$. As $f(0)=f(1)=0$, we then get that for any $p^{A}$ we have

[TABLE]

which means that

[TABLE]

Taking now $p^{A}_{i}=\delta_{i1}$ and an arbitrary $p^{B}$ we similarly obtain

[TABLE]

which together with (18) ensures that the only function $\Phi(x,y)$ of the form (17) that one can have here is

[TABLE]

with $\alpha$ a real constant.

Step 2: The general form of $f(t)$ is that of Tsallis entropy

To find the expression for $f$, let us consider variations in the identity (16), just as in the proof of Lemma 3. Indeed, substituting the probability density $p^{A}$ by the curve $p(s)$ defined in (9) and differentiating at zero, we obtain that

[TABLE]

Here we have used that the function $\Phi$ is given by (19). As the constants $P_{l}$ are arbitrary, this shows that for all $1\leqslant l\leqslant W-1$ one has

[TABLE]

Let us now consider variations with respect to the probability density $p^{B}$. Just as in the proof Lemma (3), let us assume that $p^{B}$ is not on the boundary of the set of $W^{\prime}$-state probability densities $\mathcal{P}_{W^{\prime}}$ (i.e., $p_{j}^{B}>0$ for all $1\leqslant j\leqslant W^{\prime}$). (We recall that the set $\mathcal{P}_{W^{\prime}}$ was defined in (8)). Then one can take a small enough $\varepsilon$ such that the curve

[TABLE]

is contained in $\mathcal{P}_{W^{\prime}}$ for all $s\in(-\varepsilon,\varepsilon)$ and each vector

[TABLE]

in $\mathbb{R}^{W^{\prime}-1}$ with $|\bar{P}|\leqslant 1$.

Evaluating the identity (20) on $p^{B}=\bar{p}(s)$ and differentiating at $s=0$ we then get that for all $1\leqslant l\leqslant W-1$ one has

[TABLE]

Since this holds for all $\bar{P}$ in the unit ball of $\mathbb{R}^{W^{\prime}-1}$, this ensures that for all $1\leqslant l\leqslant W-1$ and $1\leqslant m\leqslant W^{\prime}-1$ one has

[TABLE]

Without loss of generality let us take $W\geqslant 3$ to ensure that the variables $p_{i}^{A},p_{W}^{A}$ are independent. To transform the functional equation (23) into a differential equation, it is convenient to evaluate this identity on the probability distribution $p^{B}_{m}=\delta_{m1}$. (Notice that, although we have used that $p^{B}_{j}>0$ for all $j$ to derive the equation, by continuity it must also hold for this choice of $p^{B}$.) Let us fix a certain $l$ and write

[TABLE]

Equation (23) then reads as

[TABLE]

where we have used the fact that $f^{\prime}$ is continuous up to the endpoints of the interval $[0,1]$ to set

[TABLE]

Since $t$ and $\tau$ are independent, Equation (24) implies that

[TABLE]

where $c$ is a constant. For $q>0$, the solution of this equation is given in terms of two arbitrary constants as

[TABLE]

so the conditions $f(0)=f(1)=0$ then imply that

[TABLE]

where the normalization constant $c$ remains arbitrary. The case $q\leqslant 1$ leads to functions that are not of class $C^{1}$ at [math]. Theorem (1) then follows.

4. Proof of Theorem 2

In this section we present the proof of Theorem 2, which relies on the same kind of ideas as that of Theorem 1. Just as before, it is convenient to divide the proof in two steps:

Step 1: Derivation of the composition law

The starting point is the composability condition (1), which using the notation (6) and the form of the metric one can write as

[TABLE]

As the function $g$ has a $C^{1}$ inverse $g^{-1}$ because $g^{\prime}\neq 0$, one can define the $C^{1}$ function

[TABLE]

in terms of which the above relation reads as

[TABLE]

Since this identity holds true for any probability densities $p^{A}$ and $p^{B}$, Lemma 3 then ensures that there are constants $a_{j}$ such that

[TABLE]

To compute the values of the constants $a_{j}$, let us now evaluate the identity (26) when $p^{B}_{j}=\delta_{j1}$. Since $h(0)=0$, letting

[TABLE]

we then obtain that

[TABLE]

for any probability density $p^{A}$, that is,

[TABLE]

If we now take the probability density $p^{A}_{i}=\delta_{i1}$ and an arbitrary $p^{B}$, we analogously arrive at

[TABLE]

A straightforward computation then shows that the only functions of the form (27) that satisfy (28) and (29) are

[TABLE]

where $\alpha$ is an arbitrary real constant. This shows that the composition law is

[TABLE]

Step 2: The form of the function $h$

Here we shall proceed by taking variations just as in Step 2 of the proof of Theorem 1. Let us sketch the details.

First we take variations with respect to the probability density $p^{A}$, so we replace $p^{A}$ by the curve $p(s)$ (cf. Equation (9)) in the identity (26). Taking derivatives at $s=0$ and using the explicit form of $\widetilde{\Phi}$ (Equation (30)) we obtain that

[TABLE]

As the constants $P_{l}$ are arbitrary, this shows that for all $1\leqslant l\leqslant W-1$ one has

[TABLE]

This is the analog of Equation (20).

Now we take variations with respect to $p^{B}$ in (31). That is, we replace $p^{B}$ by the curve $\bar{p}(s)$ introduced in (21) and take the derivative at $s=0$ to obtain that for all $1\leqslant l\leqslant W-1$ one has

[TABLE]

This is exactly Equation (32), but with $h$ playing the role of $f$. Hence we infer from Equation (25) that $h$ must be of the form

[TABLE]

where $a,b,c$ are real constants. As $h(0)=0$, we must have $c=0$, which completes the proof of the theorem.

5. The cases of absolutely continuous or analytic functions

In this section we will present a couple of remarks about the regularity assumptions in our results. As the proofs of both theorems involve essentially the same ideas, for concreteness we will make this remarks only in the context of the first theorem (that is, trace-form entropies); the extension to entropies of the form (5) is straightforward.

We have determined the form of the composition law, which is $\Phi(x,y)=x+y+\alpha xy$ by means of Theorem 1 and is given by (30) in Theorem 2) through Lemma 3. The first observation is that this lemma holds under considerably weaker regularity assumptions, and that in fact we do not even need to assume that the composability condition holds for all probability densities: if it holds in any open subset of the space of probability densities (for example), for densities that are close enough to the uniform distributions $p^{A}_{i}=1/W$, $p^{B}_{j}=1/W^{\prime}$), the argument goes through. More precisely, we have the following:

Proposition 4**.**

Let $S$ be an entropy of the form (3) satisfying the composability condition (1) in a small neighborhood of any two probability densities $\hat{p}^{A}$ and $\hat{p}^{B}$. If $f$ is absolutely continuous on $[0,1]$ and $\Phi$ is of class $C^{1}$, then in the neighborhood under consideration the composition law must be $\Phi(x,y)=x+y+\alpha xy$ for some real constant $\alpha$.

Proof.

Since the argument we used in Lemma (3) to derive the differential equations from functional relations is purely local (because it relies on taking curves $p(s)$ with $s\in(-\varepsilon,\varepsilon)$), it is clear that the fact that the composability condition only holds in an open set does not constitute a problem.

The only aspect that one must control to derive the proposition is to ensure that the proof of Lemma 3 also remains valid under the weaker regularity assumption that $f\in AC([0,1])$ (of course, our function $f$ will play the role of the function $F$ in the lemma). The key point is to make sense of Equation (11) (or, equivalently, (12)). This is formally the derivative at $s=0$ of the map given by the left hand side of (10), so our goal is to make sense of it as a differentiable function of $s$. This is not hard, but it does not follow from a general distribution-theoretical argument either.

To this end, recall that the derivative of an absolutely continuous function is $f^{\prime}\in L^{1}((0,1))$. The key feature is that in Equation (11) one does not have to deal with functions of the form, say, $F^{\prime}(p_{i}^{A})$, but with $p_{j}^{B}\,F^{\prime}(p_{i}^{A}p_{j}^{B})$. This is important because, although the fact that a function $h(t)$ is in $L^{1}((0,1))$ does not imply that $h(xy)$ is in $L^{1}((0,1)\times(0,1))$ (this can be readily seen by taking $h(t):=\frac{1}{t}(\ln\frac{2}{t})^{-2}$, for instance), setting $z:=xy$ one sees that

[TABLE]

This shows that $x\,h(xy)\in L^{1}((0,1)\times(0,1))$ whenever $h(t)\in L^{1}((0,1))$.

Getting back to our problem and recalling that

[TABLE]

can be written in terms of the remaining $W-1$ (respectively $W^{\prime}-1$) components of the vector, we then infer that with $p_{W}^{A}$ and $p_{W^{\prime}}^{B}$ given by (33),

[TABLE]

defines a map

[TABLE]

which depends linearly on the parameters $P_{l}$. Of course, here we are using that the function

[TABLE]

is continuous, so its product with an $L^{1}$ function is in $L^{1}$. Hence, with $s\in(-\varepsilon,\varepsilon)$, the right hand side of Equation (10) defines a map $g(s)$ such that

[TABLE]

with derivative $g^{\prime}(0)=G$. From this it stems that the proof of Step 1 does remain valid for a general $f\in AC([0,1])$. ∎

The second observation is that the proof of Step 2 in Theorem 1 (that is, passing from the identity $\Phi(x,y)=x+y+\alpha xy$ to the general form of $f(t)$) becomes much easier if we assume that $f$ is are analytic in $[0,1]$ (of course, this will not be case in general: in fact, the function appearing in the Tsallis entropy is only analytic at [math] when $q$ is an integer greater than or equal to 2).

In this simple case, the fact that necessarily $f(t)=c(t-t^{q})$ for some constants $c$ and $q$ can be derived from a general result on functional equations due to Aczel [2]. Actually, a simple proof of this can be given directly from the composition equation

[TABLE]

We will present it for the benefit of the reader. We start by considering the uniform probability distributions

[TABLE]

with arbitrarily large numbers of states $W,W^{\prime}$. We then have that, setting $h(t):=f(t)/t$,

[TABLE]

As the sequences $(W^{-1})_{W=1}^{\infty}$ and $(W^{\prime-1})_{W^{\prime}=1}^{\infty}$ tend to [math], the analyticity of $h$ on $[0,1]$ implies that for all $s,t\in[0,1]$ one has

[TABLE]

Hence one can differentiate with respect to the variable $s$ and evaluate at $s=1$ to find

[TABLE]

with $\gamma:=h^{\prime}(1)$, which can be readily integrated to obtain that

[TABLE]

Since $h(1)=0$, this readily gives

[TABLE]

as we wanted to prove.

Acknowledgements

A.E. is supported by the ERC Starting Grant 633152 and by the ICMAT–Severo Ochoa grant SEV-2015-0554 (MINECO). P. T. has been partly supported by the research project FIS2015-63966, MINECO, Spain, Spain, and by the ICMAT Severo Ochoa grant SEV-2015-0554.