Intrinsic entropies of log-concave distributions

TL;DR

This paper introduces the concept of intrinsic entropies for log-concave distributions, revealing how intrinsic volumes of typical sets grow exponentially and defining a continuous family of entropy measures.

Contribution

It defines the intrinsic entropy function for log-concave distributions and proves its continuity, extending the classical notion of entropy through intrinsic volumes.

Findings

Intrinsic entropy $h_X( heta)$ grows exponentially with dimension.

$h_X( heta)$ is continuous over [0,1].

Interpolates between 0 and the classical entropy $h(X)$.

Abstract

The entropy of a random variable is well-known to equal the exponential growth rate of the volumes of its typical sets. In this paper, we show that for any log-concave random variable $X$, the sequence of the $\lfloor n\theta \rfloor^{\text{th}}$ intrinsic volumes of the typical sets of $X$ in dimensions $n \geq 1$ grows exponentially with a well-defined rate. We denote this rate by $h_X(\theta)$, and call it the $\theta^{\text{th}}$ intrinsic entropy of $X$. We show that $h_X(\theta)$ is a continuous function of $\theta$ over the range $[0,1]$, thereby providing a smooth interpolation between the values 0 and $h(X)$ at the endpoints 0 and 1, respectively.

Equations

$$X\oplus Y:=\{x+y|x\in X,y\in Y\}.$$

$$h((1-\lambda )x+\lambda y)\ge f(x{)}^{1-\lambda }g(y{)}^{\lambda }.$$

$${\int }_{{\mathbb{R}}^n}h(x)dx\ge {\left({\int }_{{\mathbb{R}}^n}f(x)dx\right)}^{1-\lambda }{\left({\int }_{{\mathbb{R}}^n}g(x)dx\right)}^{1-\lambda }.$$

$$e^{2h(X+Y)/n}\ge e^{2h(X)/n}+e^{2h(Y)/n}.$$

$$\partial(X):=\lim_{\epsilon\to 0_{+}}\frac{|X\oplus\epsilon B|-|X|}{\epsilon}=\frac{d}{d\epsilon}|X\oplus\epsilon B|\Big{|}_{0},$$

$$\lim_{\epsilon\to 0_{+}}\frac{h(X+\sqrt{\epsilon}Z)-h(X)}{\epsilon}=\frac{d}{d\epsilon}h(X+\sqrt{\epsilon}Z)\Big{|}_{0}\stackrel{{\scriptstyle(a)}}{{=}}\frac{1}{2}J(X),$$

$$|\tilde{X}\oplus \tilde{Y}|\le C(|\tilde{X}{|}^{1/n}+|\tilde{Y}{|}^{1/n}),$$

$$e^{2h(\tilde{X}+\tilde{Y})/n}\le C\left(e^{2h(\tilde{X})/n}+e^{2h(\tilde{Y})/n}\right),$$

$$|X\oplus t{B}_n|=\sum _{j=0}^n{V}_{n-j}(X){\omega }_j{t}^j,$$

$$V_i(X\cup Y)=V_i(X)+V_i(Y)-V_i(X\cap Y).$$

$$\delta (X,Y):=\max \left\{\underset{x\in X}{\sup }\underset{y\in Y}{\inf }||x-y||,\underset{y\in Y}{\sup }\underset{x\in X}{\inf }||x-y||\right\}.$$

$$V_i(X\oplus Y{)}^{1/i}\ge V_i(X{)}^{1/i}+V_i(Y{)}^{1/i},\text{ for all }i\ge 1.$$

$${\left(\frac{V_n(X)}{V_n(B)}\right)}^{1/n}\le {\left(\frac{V_j(X)}{V_j(B)}\right)}^{1/j}.$$

$$(f\stackrel{\sim }{+}g)(z):=\sup \left\{\underset{(x,y)\text{ s.t. }x+y=z}{\min }\{f(x),g(y)\}\right\}.$$

$${\hat{\mathcal{T}}}_n^{ϵ}:=\left\{x^n\in {\mathbb{R}}^n\mid e^{-n(h(X)+ϵ)}\le p_{X^n}(x^n)\le e^{-n(h(X)-ϵ)}\right\},$$

$$(1-ϵ)e^{n(h(X)-ϵ)}\le |{\hat{\mathcal{T}}}_n^{ϵ}|\le e^{n(h(X)+ϵ)}.$$

$${\mathcal{T}}_n^{ϵ}$$

$$=\left\{x^n\in {\mathbb{R}}^n|\sum _{i=1}^n\Phi (x_i)\le n(h(X)+ϵ)\right\}.$$

$$\Phi (x)=\{\begin{array}{ll}0& \text{ if }x\in (0,1),\\ +\infty & \text{ otherwise},\end{array}$$

$$\Phi (x)=\{\begin{array}{ll}0& \text{ if }x\in [0,1],\\ +\infty & \text{ otherwise}\end{array}$$

$$h(X)=\underset{ϵ\to 0_{+}}{\lim }\underset{n\to \infty }{\lim }\frac{1}{n}\log {\mu }_n^{ϵ}(n).$$

$$h_{\theta }(X):=\underset{ϵ\to 0_{+}}{\lim }\underset{n\to \infty }{\lim }\frac{1}{n}\log {\mu }_n^{ϵ}(\lfloor n\theta \rfloor ),$$

$$V_j\left(B_n(\sqrt{n\nu (1+2ϵ)})\right)=\left(\genfrac{}{}{0ex}{}{n}{j}\right)\frac{{\omega }_n}{{\omega }_{n-j}}{\left(n\nu (1+2ϵ)\right)}^{j/2},$$

$$h_X(\theta )=H(\theta )+\frac{\theta }{2}\log 2\pi e\nu +\frac{1-\theta }{2}\log (1-\theta ),$$

$$V_j\left([0,A{]}^n\right)=\left(\genfrac{}{}{0ex}{}{n}{j}\right)A^j.$$

$$h_X(\theta )=H(\theta )+\theta \log A.$$

$$h_X(\theta ):=\underset{ϵ\to 0_{+}}{\lim }\underset{n\to \infty }{\lim }\frac{1}{n}\log {\mu }_n^{ϵ}(\lfloor n\theta \rfloor ).$$

$${\mathcal{T}}_m^{ϵ}\times {\mathcal{T}}_n^{ϵ}\subseteq {\mathcal{T}}_{m+n}^{ϵ},\text{ for all }m,n\ge 1.$$

$$\sum _{i=1}^m\Phi (x_i)\le m(h(X)+ϵ)\text{ and }\sum _{i=1}^n\Phi (y_i)\le n(h(X)+ϵ).$$

$$\sum _{i=1}^{m+n}\Phi (z_i)\le (m+n)(h(X)+ϵ),$$

$$V_j(X\times Y)=\sum _{i=0}^j{V}_i(X)V_{j-i}(Y).$$

Full text

Keywords: Typical sets, log-concave random variables, differential entropy, intrinsic volumes

Intrinsic entropies of log-concave distributions

[TABLE]

1 Introduction

Convex geometry and log-concave functional analysis have been studied in tandem for many decades. Log-concave functions (or measures) form a convenient stepping stone between geometry and analysis — the indicator function of a convex set is log-concave, so one may study convex sets by studying the larger class of log-concave measures. Furthermore, log-concave measures enjoy many advantages over convex sets, including closure under convolution, making them amenable to analysis. Ball [1] carried out the first geometric study of log-concave functions, connecting isotropic measures to isotropic convex sets and generalizing several convex geometric inequalities to log-concave measures. The idea of extending and reinterpreting convex geometric results in terms of log-concave measures has since then taken root, and the term “geometrization of probability (or analysis)” coined by Milman [27] has been used to describe this approach. Milman [27] and Klartag & Milman [20] noted that developing a structured theory of the geometry of log-concave measures could have important consequences for problems that are purely geometric in nature [20, 4].

In a recent survey, Madiman et al. [22] identified two approaches that are employed to geometrize analysis: functional (or integral) lifting and probabilistic (or entropic) lifting. To illustrate these liftings, we first state the famous Brunn-Minkowski inequality in convex geometry [15]:

Theorem 1.1** (Brunn-Minkowski inequality for compact convex sets).**

Let $0<\lambda<1$, and let $X$ and $Y$ be compact convex sets. Let $|X|$ and $|Y|$ denote the $n$-dimensional volumes of $X$ and $Y$. Define the Minkowski sum of $X$ and $Y$ as

[TABLE]

Then the following equivalent statements hold:

1. (i)

$|(1-\lambda)X\oplus\lambda Y|\geq|X|^{1-\lambda}|Y|^{\lambda}$, ** and** 2. (ii)

$|(1-\lambda)X\oplus\lambda Y|^{1/n}\geq(1-\lambda)|X|^{1/n}+\lambda|Y|^{1/n}.$**

Here and in the rest of the paper $:=$ denotes equality by definition. Integral and entropic lifting can be understood with regards to the Brunn-Minkowski inequality as follows:

Integral lifting: Madiman et al. [22] classify this approach as one that replaces convex sets by log-concave functions and replaces the volume functional by the integral. The Prékopa-Leindler inequality [28, 21, 29, 15], which serves as an integral lifting of the Brunn-Minkowski inequality (BMI), as stated in (i):

Theorem 1.2** (Prékopa-Leindler inequality).**

Let $0\leq\lambda<1$, and let $f,g,$ and $h$ be non-negative integrable functions on $\mathbb{R}^{n}$ satisfying

[TABLE]

Then

[TABLE]

Apart from visual similarity, this integral lifting indeed provides a generlization; i.e., one can prove the Brunn-Minknowski inequality using the Prékopa-Leindler inequality by replacing the functions $f$, $g$, and $h$ by $1_{X}$, $1_{Y}$, and $1_{(1-\lambda)X+\lambda Y}$, respectively. 2. 2.

Entropic lifting: As per Madiman et al. [22], one replaces convex sets by random variables and the volume functional by entropy. The entropy power inequality (EPI) [32, 33, 3] serves as an entropic lifting of the Brunn-Minkowski inequality:

Theorem 1.3** (Entropy power inequality).**

Let $X$ and $Y$ be random variables on $\mathbb{R}^{n}$ with well-defined entropies $h(X)$ and $h(Y)$. Then the following inequality holds:

[TABLE]

Here, we may observe a striking similarity to the Brunn-Minkowski inequality (ii). Unlike the integral generalization, however, the EPI is not readily proven using the BMI, or vice versa, and neither can be said to be a generalization of the other.

Our work in this paper decidedly has a “geometrization or probability” flavor, and although it does not fall squarely in either of the above classifications, it is closer in spirit to the approach of entropic lifting of convex geometry. Several instances of entropic liftings have been studied in the literature, and we provide a few examples to highlight the scope of such liftings:

Surface area and Fisher information: The surface area of a compact, convex set $X\subseteq\mathbb{R}^{n}$ is defined as the limit

[TABLE]

where $B$ is the Euclidean ball of unit radius. To obtain an entropic lifting, one may replace the convex set by a random variable $X$ and replace volume by entropy. Instead of a Minkowski sum with an $\epsilon$-ball, we use a Gaussian random variable with variance $\epsilon$:

[TABLE]

where $Z\sim{\cal N}(0,1)$ and $J(X)$ is the Fisher information of $X$. The equality in step $(a)$ is known in the literature as De Bruijn’s identity [33, 11]. This indicates that the entropic lifting of surface area is Fisher information. 2. 2.

Isoperimetric inequality: Among all compact convex shapes with a fixed surface area, the Euclidean ball has the largest volume [31]. The entropic lifting of this result—that among all distributions with a fixed Fisher information, the Gaussian distribution has the maximum entropy—holds [11]. 3. 3.

Concavity of entropy power: If $X\subseteq\mathbb{R}^{n}$ is a compact, convex set, the function $f(t):=|X+tB|^{1/n}$ is concave on $t\geq 0$. This follows from a straightforward application of the Brunn-Minkowski inequality. The entropic lifting is known as Costa’s EPI [10, 12, 35], or concavity of entropy power, and states the following: If $X$ is a random variable on $\mathbb{R}^{n}$ and $Z\sim{\cal N}(0,I)$, then the function $e^{2h(X+\sqrt{t}Z)/n}$ is concave on $t\geq 0$. 4. 4.

Reverse BMI and EPI: The reverse BMI discovered by Milman [26] states that given two convex sets $X$ and $Y$, volume-preserving linear transformations exist mapping $X$ to $\tilde{X}$ and $Y$ to $\tilde{Y}$, such that the following inequality holds:

[TABLE]

where $C$ is an absolute constant. Bobkov & Madiman [5] showed that for log-concave (or more generally $\kappa$-concave, see [5] for a definition) random variables $X$ and $Y$, entropy-preserving linear transformations exist mapping random variables $X$ and $Y$ to $\tilde{X}$ and $\tilde{Y}$, such that

[TABLE]

for an absolute constant $C$.

Our work in this paper concerns the entropic liftings of what are called intrinsic volumes in convex geometry. This is closely related to the liftings 1 and 2 mentioned above. In 1, it is natural to examine higher-order derivates with respect to $\epsilon$ for further parallels. A remarkable result in convex geometry, known as Steiner’s formula [31, 19], shows that all sufficiently high-order derivatives of the volume functional are equal to 0. Since no such property holds for the derivatives of entropy, it places a limition on how far we may extend the analogy. Steiner’s formula is stated below:

Theorem 1.4** (Steiner’s formula [31, 19]).**

Let $X\subseteq\mathbb{R}^{n}$ be a compact, convex set. Let $B_{j}$ denote the $j$-dimensional Euclidean ball in $\mathbb{R}^{n}$, and let $\omega_{j}=|B_{j}|$. Then we have the following equality:

[TABLE]

where $\{V_{0}(X),V_{1}(X),\dots,V_{n}(X)\}$ are the $n+1$ intrinsic volumes of $X$.

Intrinsic volumes are functions defined on the class of compact, convex sets, and can be uniquely extended to polyconvex sets; i.e., sets that are finite unions of compact, convex sets. Some of these intrinsic volumes are known in the literature under alternate names (e.g., $V_{0}(X)$ is the Euler characteristic, $V_{1}(X)$ is the mean width, $2V_{n-1}(X)$ is the surface area, and $V_{n}(X)$ is the volume). Intrinsic volumes have a number of interpretations in geometry. We state some of these interpretations, as found in Klain & Rota [19]. Intrinsic volumes are valuations on polyconvex sets; i.e., for all polyconvex sets $X$ and $Y$ and for all $0\leq i\leq n$,

[TABLE]

Furthermore, these valuations are convex-continuous and invariant under rigid motions [31]. In fact, Hadwiger’s theorem [16, 18, 19] states that any convex-continuous, rigid-motion invariant valuation on the set of polyconvex sets is a linear combination of the intrinsic volume valuations. Here, convex-continuity is defined with respect to the topology on compact convex sets induced by the Hausdorff metric $\delta$, which measures the distance between $X$ and $Y$ according to the relation

[TABLE]

Kubota’s theorem or Crofton’s formula [19, 31] implies that the $i^{\text{th}}$ intrinsic volume $V_{i}(K)$ is proportional to the volume of a random $i$-dimensional projection or slice of $K$. Intrinsic volumes are thus defined by the geometric structure of a set and describe its global characteristics.

A number of inequalities concerning volumes carry over for intrinsic volumes. Schneider [31] notes that a complete-Brunn-Minkowski inequality exists for intrinsic volumes of compact convex sets:

[TABLE]

Furthermore, the isoperimetric inequality from point 2 holds much more generally [31]: If $B$ is the unit Euclidean ball in $\mathbb{R}^{n}$,

[TABLE]

These results pose a natural question, which is also central to our work: What is a natural analog of intrinsic volumes for log-concave measures? In other words, what is an appropriate functional or probabilistic lifting of intrinsic volumes?

Several recent papers have proposed functional analogs of intrinsic volumes. In Milman & Rotem [24, 25] and Bobkov et al. [6] the authors introduce a new notion of addition and interpret intrinsic volumes (or more generally, mixed volumes) by considering the coefficients of the version of Steiner’s formula corresponding to this notion of addition. For log-concave functions $f$ and $g$ on $\mathbb{R}^{n}$, the addition operation $f\stackrel{{\scriptstyle\sim}}{{+}}g$ is given as

[TABLE]

The authors obtain analogs of several inequalities, including the Brunn-Minkowski inequality, for intrinsic volumes defined with respect to this operation. We refer the reader to the recent survey by Colesanti [9] for further references regarding such functional liftings, as well as an alternate approach to generalizing intrinsic volumes by searching for appropriate notions of valuations on the space of log-concave functions.

In this paper, we attempt to discover an entropic lifting of intrinsic volumes for log-concave distributions. To connect a probability measure to its entropy, we use the well-known result from information theory that connects the volume of a typical set of a random variable to its entropy [11]. To be precise, given a real-valued random variable $X$ with density $p_{X}$ and differential entropy $h(X)$, one way to define its $\epsilon$-typical set, $\hat{\cal T}^{\epsilon}_{n}$ in dimension $n$ is

[TABLE]

where $p_{X^{n}}(x^{n})=\prod_{i=1}^{n}p_{X}(x_{i})$. For all sufficiently large $n$, the volume $|\hat{\cal T}^{\epsilon}_{n}|$, satisfies [11]

[TABLE]

Thus, the exponential growth rate of the volume $|\hat{\cal T}^{\epsilon}_{n}|$ is determined by the differential entropy $h(X)$. Extending this analogy, we postulate that the intrinsic volumes of $\{\hat{\cal T}^{\epsilon}_{n}\}_{n\geq 1}$, or the exponential growth rate of the same sequence, yields analogs of intrinsic volumes for distributions. To ensure that typical sets have well-defined intrinsic volumes, we focus our attention on the one-sided typical sets of log-concave random variables. We formally define log-concave distributions and their typical sets, as follows:

Definition 1.1** (Log-concave distribution).**

The distribution $p_{X}$ of a real-valued random variable $X$ is said be log-concave if there exists a lower-semicontinuous convex function $\Phi:\mathbb{R}\to\mathbb{R}\cup\{+\infty\}$ such that $p_{X}(x)=e^{-\Phi(x)}$ for all $x\in\mathbb{R}$.

Definition 1.2**.**

Let $X$ be a real-valued random variable with a log-concave density $p_{X}(X):=e^{-\Phi(x)}$. For each $n\geq 1$ and $\epsilon>0$, the one-sided $\epsilon$-typical set is given by

[TABLE]

Remark 1.1**.**

For a lower semi-continuous function $\Phi$, it is possible to define a closed set as the support of $p_{X}$, such that $p_{X}$ is continuous on this support. The restriction to lower-semicontinuous functions is for technical reasons and is not crucial to our analysis. For example, the function defined by

[TABLE]

corresponding to the uniform distribution on $(0,1)$, is not a lower semi-continuous convex function. However, the typical sets of a uniform distribution on $(0,1)$ are essentially the same as those of a uniform distribution on $[0,1]$ Thus, we may consider the lower-semicontinuous version of the function

[TABLE]

in our analysis of typical sets.

Since ${\cal T}^{\epsilon}_{n}$ is a level set of the lower semi-continuous function $\sum_{i=1}^{n}\Phi(x_{i})$, it is immediate that typical sets are compact and convex. Importantly, this implies they have well-defined intrinsic volumes. Denote the intrinsic volumes of ${\cal T}^{\epsilon}_{n}$ by $\{\mu^{\epsilon}_{n}(0),\dots,\mu^{\epsilon}_{n}(n)\}$. As noted earlier, the $n^{\text{th}}$ intrinsic volume is simply the volume, and its exponential growth rate is determined by the differential entropy, since inequality (3) continues to hold for ${\cal T}^{\epsilon}_{n}$. In other words,

[TABLE]

Our analog of intrinsic volumes is obtained by taking the limit

[TABLE]

for $\theta\in[0,1]$. We refer to this function as $h_{\theta}(X)$ or $h_{X}(\theta)$, based on which parameter is considered to be fixed in the specific context. Note that unlike earlier works, this yields an entire continuum of “intrinsic volumes,” which we also refer to as intrinsic entropies. For $\theta=0$ and $1$, we observe that $h_{0}(X)=0$ and $h_{1}(X)=h(X)$. For values of $\theta\in(0,1)$, the existence of the limit as defined in equation (6) is far from obvious. For some special cases, however, we can evaluate intrinsic entropies for all values of $\theta$:

Example 1**.**

Let $X\sim{\cal N}(0,\nu)$. The one-sided $\epsilon$-typical set in this case is simply the $n$-dimensional ball of radius $n\nu(1+2\epsilon)$, denoted by $B_{n}\left(\sqrt{n\nu(1+2\epsilon)}\right)$. The intrinsic volumes of such a ball admit a closed-form expression [19], and the $j^{\text{th}}$ intrinsic volume is given by

[TABLE]

where $\omega_{i}$ is the volume of the $i$-dimensional unit ball. Substituting $j=\lfloor n\theta\rfloor$ and taking the desired limits yields

[TABLE]

where $H(\theta)=-\theta\log\theta-(1-\theta)\log(1-\theta)$ is the binary entropy function.

Example 2**.**

Let $X$ be a random variable distributed uniformly in the interval $[0,A]$. For all $\epsilon>0$, the one-sided $\epsilon$-typical set for $X$ is the $n$-dimensional cube $[0,A]^{n}$. The $j^{\text{th}}$ intrinsic volume of this cube [19] is given by

[TABLE]

Substituting $j=\lfloor n\theta\rfloor$ and taking the desired limits gives

[TABLE]

Note that in the above examples, not only does the limit exist for all values of $\theta\in[0,1]$, but it is also continuous as a function of $\theta$. For an arbitrary log-concave distribution, such an explicit calculation is not possible, as the intrinsic volumes of its typical sets are not available in closed form. Even so, the main result of this paper establishes that similar properties hold for all log-concave distributions:

Theorem 1.5**.**

Let $X\sim p_{X}$ be a log-concave random variable. Then there exists a continuous function $h_{X}:[0,1]\to\mathbb{R}$ such that for all $\theta\in[0,1]$,

[TABLE]

The remainder of this paper is structured as follows: In Section 2, we produce a candidate function $-\Lambda^{*}$, which is a continuous function on $[0,1]$, and show in the subsequent sections that it equals $h_{X}$. We break up the proof of Theorem 1.5 into three parts, A, B, and C as follows: In Section 3, we show part A, that $h_{X}(\theta)=-\Lambda^{*}(\theta)$ for $\theta\in(0,1)$. In Section 4, we show part B, that $h_{X}(0)=-\Lambda^{*}(0)$. In Section 5, we show part C, that $h_{X}(1)=-\Lambda^{*}(1)$. We conclude the paper in Section 6 by discussing future work and open problems.

2 A candidate function

Let $X$ be a real-valued random variable with a log-concave density $p_{X}(X)$, given by $p_{X}(x)=e^{-\Phi(x)}$, for a convex function $\Phi:\mathbb{R}\to\mathbb{R}\cup{+\infty}$. Our first lemma establishes a “super-multiplicative” property of the sequence of typical sets as defined in equation (4):

Lemma 2.1**.**

The sequence of sets $\{{\cal T}^{\epsilon}_{n}\}_{n\geq 1}$ satisfies

[TABLE]

Proof.

Let $x^{m}\in{\cal T}^{\epsilon}_{m}$ and $y^{n}\in{\cal T}^{\epsilon}_{n}$. We have

[TABLE]

Adding the above inequalities, $z^{m+n}=(x^{m},y^{n})$ satisfies

[TABLE]

which implies that $z^{m+n}\in{\cal T}^{\epsilon}_{m+n}$. ∎

This super-multiplicative growth of typical sets is a geometric property of typical sets. Therefore, it is unsurprising that one may glean some information about the intrinsic volumes of typical sets from such a geometric property. We state the key results that we need, the full details of which may be found in Klain & Rota [19]:

Theorem 2.1** (Properties of intrinsic volumes).**

Let $X$ and $Y$ be any compact, convex sets. Then the following statements hold:

1. (a)

If $X\subseteq Y$, then $V_{j}(X)\leq V_{j}(Y)$ for all $j\geq 0$; i.e., intrinsic volumes are monotonic with respect to inclusion. 2. (b)

The intrinsic volumes of the Cartesian product $X\times Y$ are obtained by convolving the intrinsic volumes of $X$ and $Y$; i.e., for all $j\geq 0$,

[TABLE]

Lemma 2.1 and Theorem 2.1 immediately imply the following result:

Lemma 2.2**.**

For $\epsilon>0$, let the one-sided $\epsilon$-typical sets of a log-concave random variable be $\{{\cal T}^{\epsilon}_{n}\}_{n\geq 1}$, and let the sequence of intrinsic volumes of these typical sets be $\{\mu^{\epsilon}_{n}(\cdot)\}_{n\geq 1}$. Then the sequence of intrinsic volumes satisfies the following:

[TABLE]

where $``\star"$ stands for function convolution.

Remark 2.1**.**

Note that in the above statement, we define $\mu^{\epsilon}_{n}(j)=0$ for $j\geq n+1$ to obtain the function $\mu^{\epsilon}_{n}:\mathbb{Z}_{+}\to\mathbb{R}$ starting from the finite sequence $\{\mu^{\epsilon}_{n}(0),\dots,\mu^{\epsilon}_{n}(n)\}$. This is also geometrically accurate, since higher-dimensional intrinsic volumes of a lower-dimensional set are equal to 0.

The analysis of sequences that satisfy property (15) is central to our work, and we devote the next subsection to this topic.

2.1 Super-convolutive sequences

We define sequences that satisfy a property such as inequality (15) as super-convolutive sequences:

Definition 2.1**.**

For each $n\geq 1$, let $\mu_{n}:\mathbb{Z}_{+}\to\mathbb{R}_{+}$ be such that $\mu_{n}(i)=0$, for all $i\geq n+1$. The sequence $\{\mu_{n}(\cdot)\}_{n\geq 1}$ is super-convolutive if

[TABLE]

We now turn to the analysis of super-convolutive sequences, with an emphasis on their convergence properties. First, we develop some notation and make some small observations:

Lemma 2.3**.**

Let $\{\mu_{n}(\cdot)\}$ be as in Definition 2.1. Define the sequence of generating functions $\{G_{n}\}_{n\geq 1}$ and $\{g_{n}\}_{n\geq 1}$ as follows:

[TABLE]

The following results hold:

1. (i)

For all $m,n\geq 1$ and for all $t$,

[TABLE] 2. (ii)

For all $t$, the following limit, denoted by $\Lambda(t)$, exists (although it may be $+\infty$):

[TABLE]

Proof of Lemma 2.3.

Statement (i) is immediate from inequality (16), whereas (ii) follows from a direct application of the super-additive convergence theorem, also known as Fekete’s Lemma [34]. ∎

A super-convolutive sequence $\{\mu_{n}\}$, without any other conditions imposed on it apart from condition (16), can blow up to infinity arbitrarily fast. To prevent such scenarios, we define a proper super-convolutive sequence as follows:

Definition 2.2**.**

A super-convolutive sequence as in Definition 2.1 is said to be proper if it satisfies the following conditions:

1. (i)

For all $n$, we have $\mu_{n}(n)>0$ and $\mu_{n}(0)>0$. 2. (ii)

The limit $\beta:=\lim_{n\to\infty}\frac{1}{n}\log\mu_{n}(0)$ is finite. 3. (iii)

The value $\gamma:=\Lambda(0)$ is finite.

Remark 2.2**.**

Note that $(\mu_{m}\star\mu_{n})(0)=\mu_{n}(0)\mu_{m}(0)$. Thus, the limit in condition (ii) is assured to exist by super-additivity. Note also that $\Lambda(0)$ has a simple expression:

[TABLE]

The limit

[TABLE]

is also assured to exist, since $\mu_{m+n}(m+n)=\mu_{m}(m)\mu_{n}(n)$, and condition (iii) ensures that it is finite.

The limit function $\Lambda$ of a proper super-convolutive sequence satisfies a number of desirable properties, including $\Lambda(t)<\infty$, for all $t$. We gather some of these properties in the following lemma:

Lemma 2.4** (Proof in Appendix A.1).**

For a proper super-convolutive sequence, the limit function $\Lambda$, as defined in equation (19), satisfies the following properties:

1. (i)

$g_{1}(t)\leq\Lambda(t)\leq\max(\gamma,t+\gamma)$, for all $t$. 2. (ii)

$\Lambda$* is convex and monotonically increasing.*

Our next lemma details some important properties of $\Lambda^{*}$, the convex conjugate [7] of $\Lambda$:

Lemma 2.5** (Proof in Appendix A.2).**

Let $\Lambda$ be as in Lemma 2.4, and let $\Lambda^{*}$ be its convex conjugate. Then $\Lambda^{*}$ satisfies the following properties:

1. (i)

The domain of $\Lambda^{*}$ is $[0,1]$, and $\Lambda^{*}$ is continuous and convex on its domain. 2. (ii)

For $t\not\in\{0,1\}$, the value of $\Lambda^{*}(t)$ is given by the limit

[TABLE] 3. (iii)

For $t\in\{0,1\}$, we have the inequality

[TABLE]

Remark 2.3**.**

It is easy to check that the domain of $g_{n}^{*}$ is $[0,1]$ and that

[TABLE]

and

[TABLE]

and that $g_{n}^{*}(x)\geq-\gamma$ for all $x\in[0,1]$. Using the notation from Definition 2.2 and Remark 2.2, statement (iii) is equivalent to saying that $\Lambda^{*}(0)\leq-\beta$ and $\Lambda^{*}(1)\leq-\alpha.$ We also note that it is possible to obtain a strict inequality statement (ii), and an example is provided in Appendix A.3.

We are now in a position to state our main result concerning convergence of proper super-convolutive sequences:

Theorem 2.2**.**

Let $\{\mu_{n}\}$ be as in Definition 2.2. Define a sequence of measures $\{\mu_{n/n}\}_{n\geq 1}$ supported on $[0,1]$ by assigning point masses as follows:

[TABLE]

Let $I\subseteq\mathbb{R}$ be a closed set and $F\subseteq\mathbb{R}$ be an open set. Then

[TABLE]

Proof.

The large-deviations upper bound in inequality (22) may be established via a direct application of the Gärtner-Ellis theorem [13], stated in Appendix D, as follows: We note that $\log\sum_{j}\mu_{n}(j)=G_{n}(0):=\log s_{n}$. Define the probability measure $p_{n}:=\frac{\mu_{n/n}}{s_{n}}.$ The log moment generating function of $p_{n}$ is given by

[TABLE]

Thus,

[TABLE]

This pointwise convergence (condition $(*)$ in Appendix D) is the key condition required to apply the Gärtner-Ellis theorem. We then obtain

[TABLE]

which immediately gives

[TABLE]

proving inequality (22).

We now establish the lower bound (23). We construct a sequence $\{\hat{\mu}_{n}\}$ such that $\mu_{n}\geq\hat{\mu}_{n}$, for all $n$; i.e., $\mu_{n}$ pointwise dominates $\hat{\mu}_{n}$, for all $n$. The large-deviations lower bound for $\{\hat{\mu}_{n}\}$ will serve as a large-deviations lower bound for $\{\mu_{n}\}$. Fix $a\geq 1$. We express every $n\geq 1$ as $n=qa+r$, where $r<a$, and define

[TABLE]

where $\left({\mu_{a}}\right)^{\star q}$ is $\mu_{a}$ convolved $q$ times. Since $\{\mu_{n}\}$ is super-convolutive, this definition ensures that

[TABLE]

Define $\hat{G}_{n}(t):=\log\sum_{j=0}^{n}\hat{\mu}_{n}(j)e^{jt}.$ Then

[TABLE]

where $(a)$ is true because $G_{r}(t)$ is bounded and thus inconsequential in the limit. Applying the Gärtner-Ellis theorem for $\{\hat{\mu}_{n}\}$, and noting that $g_{a}(t)$ is differentiable, we obtain

[TABLE]

To show that $\inf_{x\in F}\left(g_{a}\right)^{*}(x)$ tends to $\inf_{x\in F}\left(\Lambda\right)^{*}(x)$ as $a\to\infty$, we establish the following lemma:

Lemma 2.6** (Proof in Appendix A.4).**

Let $\{f_{n}\}$ be a sequence of continuous convex function on $[a,b]$ converging pointwise to $f$, where $|f(x)|<\infty$ for all $x\in[a,b]$. Let $F\subseteq[a,b]$ be a relatively open set, i.e. $F$ is either an open interval $(c,d)$ with $a<c<d<b$ or $(c,b]$ with $a<c<b$ or $[a,d)$ with $a<d<b$, or $[a,b]$. Then

[TABLE]

Taking the limit as $a\to\infty$ and using Lemma 2.6, which is applicable because of the properties of $g_{n}^{*}$ given in Remark 2.3, we have

[TABLE]

where $(a)$ is a consequence of Lemma 2.5, and the fact that $F$ is an open set. This concludes the proof of Theorem 2.2. ∎

2.2 Candidate function for $h_{X}$

In Section 2.1, we developed the theory of convergence of proper super-convolutive sequences. In this section, we aim to apply this theory to the particular case of sequences of intrinsic volumes, which we denote by $\{\mu^{\epsilon}_{n}\}_{n\geq 1}$. Note that Lemma 2.2 already provides the super-convolutive property of $\{\mu^{\epsilon}_{n}\}_{n\geq 1}$. Thus, we may directly apply Lemma 2.3 to conclude the existence of a limit function $\Lambda^{\epsilon}(t)$ satisfying

[TABLE]

where

[TABLE]

Our first lemma in this section shows that $\{\mu^{\epsilon}_{n}\}_{n\geq 1}$ is a proper super-convolutive sequence:

Lemma 2.7** (Proof in Appendix A.5).**

Let $\epsilon>0$, and let $\{\mu^{\epsilon}_{n}\}_{n\geq 1}$ be the sequence of intrinsic volumes of the one-sided typical sets of a log-concave random variable. Then this sequence is a proper super-convolutive sequence; i.e., it satisfies the following three conditions:

1. (i)

For all $n\geq 1$, we have $\mu^{\epsilon}_{n}(0)>0$ and $\mu^{\epsilon}_{n}(n)>0$. 2. (ii)

The limit $\beta:=\lim_{n}\frac{\log\mu^{\epsilon}_{n}(0)}{n}$ is finite. 3. (iii)

The value of $\gamma:=\Lambda^{\epsilon}(0)<\infty$.

Remark 2.4**.**

Properties (i) and (ii) are immediate, but showing (iii) is non-trivial. Our proof proceeds by constructing a super-convolutive sequence $\{\mu^{\text{cp}}_{n}\}_{n\geq 1}$ that pointwise dominates $\{\mu^{\epsilon}_{n}\}_{n\geq 1}$. This sequence corresponds to the intrinsic volumes of a family of crosspolytopes $\{{\cal C}_{n}\}_{n\geq 1}$, which are constructed in such a way that ${\cal T}^{\epsilon}_{n}\subseteq{\cal C}_{n}$ for all $n\geq 1$. The limit $\Lambda^{\text{cp}}(0)$ may be bounded using the explicit formulae for intrinsic volumes of crosspolytopes, and this bound also serves as a bound for $\Lambda^{\epsilon}(0)$.

Applying Theorem 2.2 to the proper super-convolutive sequence $\{\mu^{\epsilon}_{n}\}_{n\geq 1}$, we conclude that the convex conjugate of $\Lambda^{\epsilon}$, denoted by $(\Lambda^{\epsilon})^{*}$, characterizes the large-deviations type convergence of $\{\mu^{\epsilon}_{n}\}_{n\geq 1}$:

Theorem 2.3**.**

Let $\{\mu^{\epsilon}_{n}\}$ be as in Lemma 2.7. Define a sequence of measures $\{\mu^{\epsilon}_{n/n}\}_{n\geq 1}$ supported on $[0,1]$ by assigning point masses as follows:

[TABLE]

Let $I\subseteq\mathbb{R}$ be a closed set and $F\subseteq\mathbb{R}$ be an open set. Then

[TABLE]

Proof.

The proof is immediate using Lemma 2.7 and Theorem 2.2. ∎

It is now natural to conjecture that

[TABLE]

To show that this limit does exist, we establish the following theorem:

Theorem 2.4**.**

Define the function $-\Lambda^{*}:[0,1]\to\mathbb{R}$ as the pointwise limit of $-\left(\Lambda^{\epsilon}\right)^{*}$, as $\epsilon\to 0_{+}$:

[TABLE]

Then $-\Lambda^{*}$ is a continuous, concave function on $[0,1]$.

Proof.

From the definition of a typical set (4), it is easy to see that for $\epsilon_{1}<\epsilon_{2}$, the corresponding typical sets satisfy ${\cal T}^{\epsilon_{1}}_{n}\subseteq{\cal T}^{\epsilon_{2}}_{n}$, for all $n\geq 1$. Using the monotonicity of intrinsic volumes with respect to inclusion, we have $-\left(\Lambda^{\epsilon_{1}}\right)^{*}\leq-\left(\Lambda^{\epsilon_{2}}\right)^{*}$. Thus, for each $\theta\in[0,1]$, the value of $-\left(\Lambda^{\epsilon}\right)^{*}(\theta)$ monotonically decreases as $\epsilon\to 0_{+}$. To ensure that the quantity does not tend to $-\infty$ and establish a pointwise convergence result, we first provide a lower bound. Fix an $\epsilon_{0}>0$. From Lemma 2.5 and Remark 2.3 we have

[TABLE]

By the concavity of $-\left(\Lambda^{\epsilon}\right)^{*}$, we obtain the linear lower bound $-\left(\Lambda^{\epsilon}\right)^{*}(\theta)\geq\theta(h(X)-\epsilon)$, for all $\theta\in[0,1]$. Thus, as $\epsilon\to 0_{+}$, the value of $-\left(\Lambda^{\epsilon}\right)^{*}$ may be uniformly lower-bounded. We then use the following lemma to conclude the proof:

Lemma 2.8** (Proof in Appendix A.6).**

Let $\{f_{n}\}$ be a sequence of continuous, concave functions on $[a,b]$, converging pointwise and in a monotonically decreasing manner to a function $f$. Then $f$ is a continuous, concave function on $[a,b]$.

∎

The function $-(\Lambda^{*})$ obtained in this fashion is the candidate function we have sought. In the subsequent sections, we show that $-(\Lambda^{*})(\theta)=h_{X}(\theta)$, for all $\theta\in[0,1]$.

3 Proof of the main theorem: Part A

Theorem 3.1** (Part A of Theorem 1.5).**

Let $-\Lambda^{*}$ be as in Theorem 2.4. The following equality holds:

[TABLE]

The key result that bridges the gap between the large-deviations type convergence in Theorem 2.3 and Theorem 3.1 is the Alexandrov-Fenchel inequality [31]. While the complete inequality is quite general, holding not just for intrinsic volumes, but also for mixed volumes, we state here a version from McMullen [23]:

Theorem 3.2** (Inequality for intrinsic volumes [23]).**

Let $X\subseteq\mathbb{R}^{n}$ be a compact, convex set, and let its intrinsic volumes be $\{\mu_{n}(0),\dots,\mu_{n}(n)\}$. Then the following inequality holds:

[TABLE]

A immediate corollary of this theorem is the following:

Corollary 3.1**.**

The sequence of intrinsic volumes is log-concave; i.e.,

[TABLE]

Proof of Theorem 3.1.

Corollary 3.1 implies the log-concavity of the sequence $\{\mu^{\epsilon}_{n}(0),\dots,\mu^{\epsilon}_{n}(n)\}$. Our goal is to show that a family of log-concave sequences such as $\{\mu^{\epsilon}_{n}(\cdot)\}$, which converges in the large-deviation sense to $-\left(\Lambda^{\epsilon}\right)^{*}$, also converges pointwise to $-\left(\Lambda^{\epsilon}\right)^{*}$ in the following sense: For $\theta\in(0,1)$,

[TABLE]

For all $n\geq 1$, define the functions $a^{\epsilon}_{n}(\theta)$ by linearly interpolating the values of $a^{\epsilon}_{n}(j/n)$, where the value of $a^{\epsilon}_{n}(j/n)$ is given by

[TABLE]

The following lemma is a simple restatement of Corollary 3.1:

Lemma 3.1** (Proof in Appendix B.1).**

For each $n$ and $\epsilon>0$, the function $a^{\epsilon}_{n}(\cdot)$ is concave.

Next, we show that the sequence of functions $\{a^{\epsilon}_{n}\}_{n\geq 1}$ converges pointwise—indeed, uniformly—on closed intervals:

Lemma 3.2** (Proof in Appendix B.2).**

Let ${\cal I}\subset(0,1)$ be a closed interval. The sequence of functions $\{a^{\epsilon}_{n}\}$ converges uniformly to $-(\Lambda^{\epsilon})^{*}(\theta)$ on ${\cal I}$.

Proof sketch.

Although not difficult, the proof of this lemma is fairly technical. For $\theta\in{\cal I}$, it is possible to upper-bound $\lim\sup_{n}a^{\epsilon}_{n}$ by a number arbitrarily close to $-\Lambda^{*}(\theta_{0})$, by a straightforward application of Theorem 2.3 to a small interval $I_{0}$ that contains $\theta_{0}$. To lower-bound $\lim\inf_{n}a^{\epsilon}_{n}(\theta)$, we trap the interval $I_{0}$ between the intervals to its left and right, denoted by $I_{-1}$ and $I_{+1}$. We show that there exist $\theta_{-1}\in I_{-1}$ and $\theta_{+1}\in I_{+1}$ such that $a^{\epsilon}_{n}$ evaluated at both these points is close to $-\Lambda^{*}(\theta_{0})$, and apply concavity provided by Corollary 3.1 to produce a lower bound for $\lim\inf_{n}a^{\epsilon}_{n}(\theta)$ that is also close to $-\Lambda^{*}(\theta_{0})$. Taking the limit, we conclude the proof of Lemma 3.2. ∎

To complete the proof of Theorem 3.1, we take the limit as $\epsilon\to 0_{+}$ and use Theorem 2.4, which produces the claimed result:

[TABLE]

∎

The above strategy does not succeed in establishing the theorem at the endpoints $\theta=0$ and $\theta=1$, however. The main difficulty is that the sandwiching argument described in the proof sketch of Lemma 3.2 can no longer work, since there is no interval to sandwich the endpoints between. To settle these cases, we exploit some additional geometric properties of typical sets in the following two sections.

4 Proof of the main theorem: Part B

Theorem 4.1** (Part B of Theorem 1.5).**

Let $-\Lambda^{*}$ be as in Theorem 2.4. The following equality holds:

[TABLE]

This is equivalent to the claim $-\Lambda^{*}(0)=0.$

Proof.

The proof of this result resembles the proof of Lemma 2.7, particularly regarding the approach to proving $\gamma<\infty$ as (cf. Remark 2.4). In Lemma 2.7, we constructed a sequence of crosspolytopes $\{{\cal C}_{n}\}$ such that ${\cal T}^{\epsilon}_{n}\subseteq{\cal C}_{n}$ for all $n\geq 1$. We briefly describe how to achieve this in the present situation. Observe that for each log-concave distribution $p_{X}(x)=e^{-\Phi(x)}$, we may find constants $c_{1}>0$ and $c_{2}$ such that

[TABLE]

For $A=\frac{h(X)+\epsilon-c_{2}}{c_{1}}$, define the sequence of regular crosspolytopes $\{{\cal C}_{n}\}_{n=1}^{\infty}$ defined by

[TABLE]

If $(x_{1},x_{2},\dots,x_{n})\in{\cal T}^{\epsilon}_{n}$, then by Definition 1.2,

[TABLE]

Thus,

[TABLE]

giving us $(x_{1},\dots,x_{n})\in{\cal C}_{n}$, and consequently, ${\cal T}^{\epsilon}_{n}\subseteq{\cal C}_{n}$.

Lemma 4.1**.**

Let the intrinsic volumes of $\{{\cal C}_{n}\}_{n\geq 1}$ be given by $\{\mu^{\text{cp}}_{n}\}_{n\geq 1}$. Then the following statements are true:

1. (i)

The sequence $\{{\cal C}_{n}\}_{n\geq 1}$ is super-multiplicative; i.e.

[TABLE] 2. (ii)

The sequence of intrinsic volumes $\{\mu^{\text{cp}}_{n}\}_{n\geq 1}$ is a proper super-convolutive sequence. 3. (iii)

There exists a continuous and concave function $-(\Lambda^{\text{cp}})^{*}$ such that for all $\theta\in[0,1)$, the following equality holds:

[TABLE]

In particular, $-(\Lambda^{\text{cp}})^{*}(0)=0$.

Proof of Lemma 4.1.

Part (i) is easily verified by following the same steps as in Lemma 2.1. The proof of part (ii) is already contained in Lemma 2.7, since we have shown that

[TABLE]

via the explicit formulae for $\mu^{\text{cp}}_{n}$. Since $\{\mu^{\text{cp}}_{n}\}_{n\geq 1}$ is a proper super-convolutive sequence, we use Theorem 2.2 to infer the existence of $-(\Lambda^{\text{cp}})^{*}$, establishing the convergence of $\{\mu^{\text{cp}}_{n}\}_{n\geq 1}$ in the large-deviations sense. Following exactly the same steps as in Theorem 3.1, we conclude the pointwise convergence

[TABLE]

Thus, the only part we need to show is $-(\Lambda^{\text{cp}})^{*}(0)=0$. As in Lemma 2.7, we exploit the fact the intrinsic volumes of crosspolytopes are available in closed form. By the continuity of $-(\Lambda^{\text{cp}})^{*}$, we have

[TABLE]

The value of $\mu_{n}^{\text{cp}}(\lfloor n\theta\rfloor)$ is given by

[TABLE]

As shown in the proof of Lemma 2.7,

[TABLE]

Thus, we obtain

[TABLE]

It is easy to verify that the right-hand side evaluates to 0, so

[TABLE]

However, by Lemma 2.5, we also have

[TABLE]

This shows that $-(\Lambda^{\text{cp}})^{*}(0)=0$ and completes the proof. ∎

The containment ${\cal T}^{\epsilon}_{n}\subseteq{\cal C}_{n}$ implies

[TABLE]

Thus, we obtain the inequality

[TABLE]

Furthermore, Lemma 2.5 implies that

[TABLE]

This forces $-(\Lambda^{\epsilon})^{*}(0)=0$. Taking the limit as $\epsilon\to 0_{+}$, we arrive at $-\Lambda^{*}(0)=0$, completing the proof. ∎

5 Proof of the main theorem: Part C

Theorem 5.1** (Part C of Theorem 1.5).**

Let $-\Lambda^{*}$ be as in Theorem 2.4. The following equality holds:

[TABLE]

This is equivalent to the claim $-\Lambda^{*}(1)=h(X).$

Proof.

To prove this part, we use the following inequality for intrinsic volumes proved in Campi & Gronchi [8]:

Theorem 5.2** (Loomis-Whitney type inequality for intrinsic volumes [8]).**

*Let $X\subseteq\mathbb{R}^{n}$ be a compact set and let its intrinsic volumes be $\{V_{0}(X),\dots,V_{n}(X)\}$. Let $\{e_{1},\dots,e_{n}\}$ be the standard basis for $\mathbb{R}^{n}$. For a set $S\subset\{1,2,\dots,n\}$, denote $e_{S}=\{e_{j}|j\in S\}$. Denote $\{1,2,\dots,n\}\setminus\{i\}$ by $\{i\}^{c}$. Let $X|S$ be the set obtained by orthogonally projecting $X$ on the space spanned by $e_{S}$.

For $0\leq m\leq n-1$, the following inequality holds:

[TABLE]

provided the intrinsic volumes of $X|\{j\}^{c}$ satisfy the stability condition

[TABLE]

Our first lemma concerns intrinsic volumes of projections of typical sets:

Lemma 5.1**.**

Let $S_{1},S_{2}\subset\{1,2,\dots,n\}$ be such that $|S_{1}|=|S_{2}|=k$. Then the intrinsic volumes of $({\cal T}^{\epsilon}_{n}\mid S_{1})$ are the same as those of $({\cal T}^{\epsilon}_{n}\mid S_{2})$. In other words, intrinsic volumes of such projections depend only on the dimension and not the specific choice among the ${n\choose k}$ possible subspaces.

Proof.

Without loss of generality, let $S_{1}=\{1,2,\dots,k\}$ and $S_{2}=\{u_{1},u_{2},\dots,u_{k}\}$. The projection $({\cal T}^{\epsilon}_{n}\mid S_{1})$ is the set

[TABLE]

Suppose $\min_{x}\Phi(x)=\eta$. Then the above expression is equivalent to

[TABLE]

Similarly,

[TABLE]

It is now clear that these two sets are exactly the same, except in different dimensions. In particular, they have identical intrinsic volumes. ∎

Our next lemma shows that Theorem 5.2 is applicable to sets of the form $({\cal T}^{\epsilon}_{n}\mid S)$:

Lemma 5.2**.**

Let $S=\{u_{1},u_{2},\dots,u_{k}\}\subseteq\{1,2,\dots,n\}$. Let $({\cal T}^{\epsilon}_{n}\mid S)$ be the set ${\cal T}_{S}$. Let $0\leq m\leq k-1$. Then ${\cal T}_{S}$ satisfies the following inequality:

[TABLE]

Proof.

We only need to check that the condition $(*)$ holds for ${\cal T}_{S}$; i.e., we need to check, for all $i$, that

[TABLE]

The sets $({\cal T}_{S}\mid S\setminus\{u_{i}\})$ are simply $(k-1)$-dimensional projections of ${\cal T}^{\epsilon}_{n}$, and by Lemma 5.1 all such sets have identical intrinsic volumes. This trivially yields inequality (46) and completes the proof. ∎

With Lemma 5.2 established, we are now in a position to iterate the inequality (43) in Theorem 5.2 until we are left with $m$-dimensional projections of ${\cal T}^{\epsilon}_{n}$. Note that each of the ${n\choose m}$ orthogonal subspaces is counted $(n-m)!$ times in the iteration. Additionally, there is a factor of $\frac{1}{(n-m)(n-m-1)\cdots 1}=\frac{1}{(n-m)!}$ outside the summation. These factors cancel out, yielding the key inequality

[TABLE]

Our final lemma produces a bound on $|({\cal T}^{\epsilon}_{n}\mid\{1,2,\dots,m\})|$:

Lemma 5.3**.**

Let $\eta=\min_{x}\Phi(x)$. Then we have the bound

[TABLE]

Proof.

Using equation (45), we have

[TABLE]

where $\eta=\min_{x}\Phi(x)$. Rewriting, we have

[TABLE]

Since the probability density over $({\cal T}^{\epsilon}_{n}\mid\{1,2,\dots,m\})$ is at most 1, we obtain the upper bound

[TABLE]

completing the proof. ∎

For $\theta\in(0,1)$, choose $m=\lfloor n\theta\rfloor$. Substituting in inequality (47) and using inequality (48), we obtain

[TABLE]

Taking logarithms of both sides and dividing by $n$, we have

[TABLE]

Taking the limit as $n\to\infty$ and using Theorem 4.1, we then have

[TABLE]

Taking the limit as $\epsilon\to 0_{+}$ and using Theorem 2.4, we obtain

[TABLE]

Taking the limit as $\theta\to 1$ and using the continuity of $-\Lambda^{*}$ from Theorem 2.4, we also have

[TABLE]

Additionally, we have the lower bound from Lemma 2.5, which asserts that

[TABLE]

This forces $-\Lambda^{*}(1)=h(X)$ and completes the proof. ∎

6 Conclusion

In this paper, we have established an entropic lifting of the concept of intrinsic volumes for the special case of log-concave distributions. For a log-concave random variable $X$, the function $h_{X}(\cdot)$ may be interpreted as a generalization of the entropy functional, in a similar way as intrinsic volumes are considered a generalization of the volume functional. We now briefly describe several future research directions and open problems.

The first natural question is whether it is possible to define intrinsic entropies for random variables that do not possess a log-concave density. Note that the main reason we focused on log-concave random distributions was that it is very straightforward to define convex typical sets for these distributions, and convex sets have well-defined intrinsic volumes. However, the concept of intrinsic volumes is not just restricted to convex sets or polyconvex sets, and may be extended to a larger class of sets. Federer [14] defined the concept of sets with positive reach, as follows: The reach of a set $A\subset\mathbb{R}^{n}$ is the largest value $r$ such that for all $x$ satisfying $d(x,A)<r$, the set $A$ contains a unique point that is nearest to $x$. Here, $d(x,A)$ is the minimum of the distance between $x$ and some point in $A$. A set has a positive reach if $r>0$. Federer [14] showed that it is possible to define intrinsic volumes for all sets having a positive reach. One possible approach to defining intrinsic entropies for distributions that are not log-concave would be to show that typical sets have a positive reach, and then take the limits of the intrinsic volumes as defined in [14]. A paper by Schanuel [30] neatly illustrates where the theory of convex sets breaks down when we consider non-convex sets and suggests ways to surmount such difficulties.

Another natural question is the following: How crucially does the value of $h_{X}(\theta)$ depend on the specific way we have defined typical sets in Definition 1.2? Suppose we had an alternate definition for a sequence of (convex) sets $\{\tilde{\cal T}^{\epsilon}_{n}\}$, satisfying the following two properties:

1. (i)

$\lim_{\epsilon\to 0}\lim_{n\to\infty}\frac{1}{n}\log|\tilde{\cal T}^{\epsilon}_{n}|=h(X)$, and 2. (ii)

$\lim_{n\to\infty}P(\tilde{\cal T}^{\epsilon}_{n})=1$.

It is interesting to ask if this sequence would exhibit the same limit for its intrinsic volumes as $\{{\cal T}^{\epsilon}_{n}\}$. One example of such an alternate sequence is as follows: Suppose $X$ satisfies $EX=0$ and $\text{Var}(X)=\sigma^{2}$. We define

[TABLE]

For such alternate definitions, the super-multiplicative property of typical sets in expression (13) may no longer hold. Thus, we may not be able to use the same analysis as presented in this paper to evaluate the limit of the intrinsic volumes. We believe the limit remains $h_{X}(\cdot)$ for any alternate definition of convex typical sets satisfying properties (i) and (ii), so it is a intrinsic property of the distribution itself. Although this problem is still open, we show in Appendix C that for many natural definitions of typical sets, intrinsic volumes grow at the same shared rate $h_{X}$.

In this paper, we have considered one-dimensional random variables. However, it is not hard to see that similar results also hold for multi-dimensional random variables. Thus, it is possible to generalize concepts such as joint entropy, conditional entropy, and mutual information by replacing entropy by intrinsic entropy in the definitions. The main focus of this paper was showing the existence of $h_{\theta}(X)$, as opposed to studying the particular properties of $h_{X}$. An important first step towards this would be to check whether intrinsic entropies satisfy an analog of the Brunn-Minkowski inequality. Here, we conjecture a version of the entropy power inequality, inspired by the complete Brunn-Minkowski inequality for intrinsic volumes [31]:

[TABLE]

where we recover the usual EPI for $\theta=1$.

Acknowledgements

The research of the authors was supported by NSF grant ECCS-1343398 and the NSF Science & Technology Center grant CCF-0939370, Science of Information. VA also acknowledges support from the NSF grants CNS 1527846 and CCF 1618145.

Appendix A Proofs for Section 2

A.1 Proof of Lemma 2.4

1. (i)

Condition (18) implies that

[TABLE]

which implies

[TABLE]

Taking the limit in $n$, we see that $g_{1}(t)\leq\Lambda(t)$.

For every $n$ and every $t\geq 0$,

[TABLE]

Taking the limit in $n$ and using $\gamma<\infty$, we see that for $\Lambda(t)\leq t+\gamma$ for $t\geq 0$. Similarly, for $t\leq 0$

[TABLE]

Taking the limit in $n$ and using $\gamma<\infty$, we see that $\Lambda(t)\leq\gamma$ for $t\leq 0$. 2. (ii)

The functions $\{g_{n}\}$ are convex and monotonically increasing. Since $\Lambda$ is the pointwise limit of these functions, $\Lambda$ is also convex and monotonically increasing.

A.2 Proof of Lemma 2.5

1. (i)

Note that the convex conjugates of the functions $g_{1}(t)$ and $\max(\gamma,t+\gamma)$ are both supported on $[0,1]$. By Lemma 2.4, the function $\Lambda$ is trapped between these two functions, and thus $\Lambda^{*}$ is also supported on $[0,1]$. Furthermore, since $\Lambda^{*}$ is a convex conjugate function, it is convex and lower-semicontinuous on its domain. Since the domain is a closed interval, we obtain that $\Lambda^{*}$ must be continuous on $[0,1]$. 2. (ii)

We’ll need to show that $g_{n}^{*}$ converges pointwise to $\Lambda^{*}$ on $(0,1)$. Fix an $x\in(0,1)$, and define

[TABLE]

Clearly, $g_{n}^{*}(x)=xt_{n}-g_{n}(t_{n})$. Note that

[TABLE]

where $(a)$ is because $\gamma=\sup_{n}g_{n}(0)$. If $t>\frac{\gamma-\log\mu_{1}(1)}{1-x}$, then we have

[TABLE]

This gives us that $t_{n}\leq\frac{\gamma-\log\mu_{1}(1)}{1-x}$. Similarly, if $t<\frac{\log\mu_{1}(0)-\gamma}{x}$, then

[TABLE]

This gives us that $t_{n}\geq\frac{\log\mu_{1}(0)-\gamma}{x}$. We can thus conclude that for all $n$,

[TABLE]

Note that all we used to prove relation (58) is that $g_{n}(t)$ is trapped between $g_{1}(t)$ and $\max(\gamma,t+\gamma)$. Since $\Lambda$ also satisfies this, we have

[TABLE]

We now restrict our attention to the compact interval $I_{x}$. Let $\hat{g}_{n}$ be $g_{n}$ restricted to $I_{x}$. The convex functions $\hat{g}_{n}$ converge pointwise to a continuous limit $\hat{\Lambda}$, where $\hat{\Lambda}$ is $\Lambda$ restricted to $I_{x}$. This convergence must therefore be uniform [17], which implies convergence of $\hat{g}_{n}^{*}(x)$ to $\hat{\Lambda}^{*}(x)$. Furthermore, relation (58) implies $\hat{g}_{n}^{*}(x)$ equals $g_{n}^{*}(x)$, and relation (59) gives $\hat{\Lambda}^{*}(x)$ equals $\Lambda^{*}(x)$. Thus, $g_{n}^{*}(\cdot)$ converges pointwise to $\Lambda^{*}(\cdot)$ on $(0,1)$. 3. (iii)

To get the inequality for $t=0$ and $t=1$, we note that the function $\lim_{n}g_{n}^{*}(t)$ and $\Lambda^{*}$ are both convex on $[0,1]$ with $\Lambda^{*}$ also being continuous on $[0,1]$. As both these function agree on $(0,1)$, it is immediate that $\Lambda^{*}(0)\leq-\beta$ and $\Lambda^{*}(1)\leq-\alpha$.

A.3 Example showing strict inequality at [math]

We show that it is possible for a proper super-convolutive sequences obey the strict inequality in Lemma 2.5 (iii), using the following example:

Example 3**.**

Let $\delta,\alpha>0$ be such that $\alpha>1$ and $\delta<\frac{1}{2}$. For $n\geq 1$, consider the sequence $\{\mu_{n}(\cdot)\}$, given by

[TABLE]

By an explicit calculation, we may show that for this sequence is a proper super-convolutive sequence, and we have the strict inequality $\Lambda^{*}(1)=-\log\alpha<0$.

A.4 Proof of Lemma 2.6

We first prove the following lemma:

Lemma A.1**.**

Let $\{g_{n}\}$ be a sequence of continuous convex function on $[c,d]$ converging pointwise to $g$, where $g$ is a continuous function. Let $F\subseteq[c,d]$ be a relatively open set. Then

[TABLE]

Proof.

By Lemma B.1, we know that $g_{n}$ converge uniformly to $g$ on $[c,d]$. Thus, for any $\epsilon>0$ we have

[TABLE]

uniformly over $x\in[c,d]$, for all $n$ sufficiently large. Let $x_{n}\in F$ be such that

[TABLE]

Such an $x_{n}$ exists by the relative openness of $F$ and the continuity of $g_{n}$ on $[c,d]$. Then, for all $n$ sufficiently large we have

[TABLE]

and we also have

[TABLE]

which completes the proof. ∎

The function $f$, being a pointwise limit of the convex functions $f_{n}$ on $[a,b]$, is convex on $[a,b]$. Since $|f(x)|<\infty$ for all $x\in[a,b]$, we may conclude that $f$ is continuous on $(a,b)$. Let $F\subseteq[a,b]$ be the relatively open subset of interest. The convexity of $f$ on $[a,b]$ implies that

[TABLE]

Let $\delta>0$ be sufficiently small that $F\cap[a+\delta,b-\delta]\neq\emptyset$. By Lemma B.1 we conclude that, for any $\delta>0$, $f_{n}$ converges uniformly to $f$ on $[a+\delta,b-\delta]$. By Lemma A.1, we conclude that

[TABLE]

Since $\inf_{x\in F\cap[a+\delta,b-\delta]}f_{n}(x)$ is nondecreasing in $\delta$, we have

[TABLE]

for all $\delta>0$. Taking the limit as $\delta\to 0$ and using (61) we get

[TABLE]

For the inequality in the opposite direction, let $M<\infty$ be such that $f_{n}(a)\leq M$ and $f_{n}(b)\leq M$ for all $n\geq 1$. Such an $M$ exists under the assumptions of Lemma 2.6. For any $\delta>0$ sufficiently small and any $b-\delta<x\leq b$, from the convexity of $f_{n}$, we have

[TABLE]

This can be rearranged to read

[TABLE]

We can similarly get a lower bound on $f_{n}(x)$ for $a\leq x<a+\delta$. We conclude that there is a universal constant $C<\infty$ such that for all $F\subset[a,b]$ that are relatively open, for $\delta>0$ sufficiently small (depending on $F$), all $n\geq 1$, all $x\in(b-\delta,b]$, and all $x\in[a,a+\delta)$, we have

[TABLE]

It follows that

[TABLE]

Fix $\delta$ and let $n\to\infty$ and use (61) to conclude that

[TABLE]

Next, let $\delta\to 0$ and use the convexity of $f$ to conclude that

[TABLE]

This completes the proof of Lemma 2.6.

A.5 Proof of Lemma 2.7

Note that $\mu^{\epsilon}_{m}\star\mu^{\epsilon}_{n}(m+n)=\mu^{\epsilon}_{n}(n)\mu^{\epsilon}_{m}(m)$ and $\mu^{\epsilon}_{m}\star\mu^{\epsilon}_{n}(0)=\mu^{\epsilon}_{n}(0)\mu^{\epsilon}_{m}(0)$. Thus the existence of the limits defining $\alpha$ and $\beta$ is given by sub-additivity. Existence of limit defining $\gamma$ follows from the equality $\gamma=\Lambda^{\epsilon}(0)$.

1. (i) and (ii):

The value of $\mu_{n}(0)$ is the Euler characteristic, which equals $1$ when ${\cal T}_{n}^{\epsilon}$ is non-empty. We show that for every $n\geq 1$, the set ${\cal T}^{\epsilon}_{n}$ has a nonempty interior; i.e., $\text{Vol}({\cal T}^{\epsilon}_{n})=\mu_{n}(n)>0$. Let $M=\max_{x}p_{X}(x)=e^{-\min_{x}\Phi(x)}$. Note that the set of minimizers of $\Phi$ is a nonempty set, since $\Phi\to+\infty$ as $|x|\to+\infty$. Let $x^{*}$ be any such minimizer of $\Phi$. For the point $(x^{*},\dots,x^{*})\in\mathbb{R}^{n}$, we have

[TABLE]

We also have the inequality

[TABLE]

Thus, for the point $(x^{*},\dots,x^{*})$, we have

[TABLE]

so $(x^{*},\dots,x^{*})\in{\cal T}_{n}^{\epsilon}$. By the continuity of $\Phi$ at $x^{*}$, we conclude that ${\cal T}_{n}^{\epsilon}$ has a nonempty interior. 2. (iii)

Since $\Phi(x)\to\pm\infty$ as $|x|\to\pm\infty$ and $\Phi$ is convex, we may find constants $c_{1}>0$ and $c_{2}$ such that

[TABLE]

We start by showing that for $A:=\frac{h(X)+\epsilon-c_{2}}{c_{1}}$, the sequence of regular crosspolytopes $\{{\cal C}_{n}\}_{n=1}^{\infty}$ defined by

[TABLE]

satisfies the containment

[TABLE]

For $x^{n}\in{\cal T}^{\epsilon}_{n}$, using definition (5) and inequality (62), we have

[TABLE]

implying that

[TABLE]

so $x^{n}\in{\cal C}_{n}$. Hence, ${\cal T}^{\epsilon}_{n}\subseteq{\cal C}_{n}$, as claimed. Let the intrinsic volumes of ${\cal C}_{n}$ be $\mu^{\text{cp}}_{n}(\cdot)$. Note that $\mu_{n}(i)\leq\mu^{\text{cp}}_{n}(i)$ for all $0\leq i\leq n$, by the containment (63). Thus, $\gamma\leq\gamma^{\text{cp}}$, where

[TABLE]

We claim that $\gamma^{\text{cp}}<\infty$. Define

[TABLE]

Note that the sequence $\{{\cal C}_{n}\}$ is super-convolutive, so $g^{\text{cp}}_{n}(t)$ converges pointwise. In particular, for $t=0$, we have

[TABLE]

The $i$-th intrinsic volume of ${\cal C}_{n}$ is given by [2]

[TABLE]

Note that

[TABLE]

Thus, for $0\leq i\leq n-1$,

[TABLE]

We may check that the inequality also holds for $i=n$. Hence,

[TABLE]

Taking the limit as $n\to\infty$, we obtain

[TABLE]

This shows that $\gamma^{\text{cp}}$ is finite, and thus $\gamma$ is also finite.

A.6 Proof of Lemma 2.8

Without loss of generality, take $a=0$ and $b=1$. Since $f$ is the pointwise limit of concave functions, it is also concave. The continuity of $f$ is not obvious a priori: it could be discontinuous at the endpoints 0 and 1. Let $f(0)=\ell_{0}$ and $f(1)=\ell_{1}$. For any $n\geq 1$, the function $f_{n}$ is lower-bounded by the line joining $(0,\ell_{0})$ and $(1,\ell_{1})$. Call this lower bound $L(\theta)$, for $\theta\in[0,1]$. We prove continuity at [math], by showing that for $\eta>0$, there exists a $\delta>0$ such that for $\theta\in[0,\delta)$, we have $|f(\theta)-\ell_{0}|<\eta$. Pick $N$ large enough such that

[TABLE]

The function $f_{N}$ is continuous on $[0,1]$, so there exists $\delta_{1}>0$ such that for $\theta\in[0,\delta_{1})$, we have $|f_{N}(\theta)-f_{N}(0)|<\eta/2$. Now pick a $\delta_{2}$ such that $|L(\theta)-\ell_{0}|<\eta/2$, for $\theta\in[0,\delta_{2})$. Let $\delta=\min(\delta_{1},\delta_{2})$. For $n>N$, we have

[TABLE]

Thus, for $\theta\in[0,\delta)$, we obtain

[TABLE]

and

[TABLE]

Thus, for all $n>N$ and $\theta\in[0,\delta)$, we have

[TABLE]

Taking the limit as $n\to\infty$, we conclude that for $\theta\in[0,\delta)$,

[TABLE]

implying continuity at [math]. Continuity at $1$ follows similarly.

Appendix B Proofs for Section 3

B.1 Proof of Lemma 3.1

To show concavity of $a_{n}(\cdot)$, note that all we need to prove is that

[TABLE]

as $a_{n}$ is a linear interpolation of the values at $\frac{j}{n}$. This is equivalent to proving

[TABLE]

This is an easy application of the Alexandrov-Fenchel inequalities for mixed volumes. For a proof we refer to McMullen [23], where in fact the author obtains

[TABLE]

B.2 Proof of Lemma 3.2

Let $\eta>0$ be given. The function $-(\Lambda^{\epsilon})^{*}$, being continuous on the bounded interval $[0,1]$, is uniformly continuous. The same holds over the closed set ${\cal I}\subset(0,1)$. Choose $\delta>0$ such that

[TABLE]

Choose $N_{0}>1/(\delta/3)$, and divide the interval ${\cal I}$ into the the $N_{0}$ equally sized closed intervals $I_{j}$, for $1\leq j\leq N_{0}$. Let the interval $I_{j}:=[c_{j-1},c_{j}]$, where $[c_{0},c_{N_{0}}]={\cal I}$. Without loss of generality, let $\theta_{0}$ lie in the interior of the $k$-th interval (we can always choose a different value of $N_{0}$ to make sure $\theta_{0}$ does not coincide with the boundary points $c_{k-1}$ or $c_{k}$). Thus,

[TABLE]

Theorem 2.2 along with the continuity of $-(\Lambda^{\epsilon})^{*}$ imply that

[TABLE]

For $n>2/\min\left(\theta_{0}-c_{k-1},c_{k}-\theta_{0}\right)$, there exists an $i$ such that

[TABLE]

Thus for some $\lambda>0$, we can write

[TABLE]

and obtain the inequality

[TABLE]

Thus we have the upper bound

[TABLE]

where $(a)$ follows from the choice of $N_{0}$ and uniform continuity of $-(\Lambda^{\epsilon})^{*}$.

Define

[TABLE]

As

[TABLE]

it is easy to see that

[TABLE]

Note that since $a^{\epsilon}_{n}$ is obtained by a linear interpolation, we have

[TABLE]

This implies that for the intervals $I_{k-1}$ and $I_{k+1}$,

[TABLE]

Since $a^{\epsilon}_{n}(\theta)$ is concave, this implies

[TABLE]

Taking the $\liminf$ on both sides,

[TABLE]

Inequalities (73) and (81) prove the pointwise convergence of $a^{\epsilon}_{n}(\theta_{0})$ to $-(\Lambda^{\epsilon})^{*}(\theta_{0})$. We then use the following lemma to conclude uniform convergence:

Lemma B.1**.**

Let $\{f_{n}\}$ be a sequence of continuous convex functions which converge pointwise to a continuous function $f$ on an interval $[a,b]$. Then $f_{n}$ converge to $f$ uniformly.

Proof.

Let $\epsilon>0$. We’ll show that there exists a large enough $N$ such that for all $n>N$, $||f_{n}-f||_{\infty}<\epsilon$.

The function $f$ is continuous on a compact set, and therefore is uniformly continuous. Choose a $\delta>0$ such that $|f(x)-f(y)|<\epsilon/10$ for $|x-y|<\delta$. Let $M$ be such that $(b-a)/M<\delta$. We divide the interval $[a,b]$ into $M$ intervals, whose endpoints are equidistant. We denote them by $a=\alpha_{0}<\alpha_{1}<\cdots<\alpha_{M}=b$. Since $f_{n}(\alpha_{i})\to f(\alpha_{i})$, there exists a $N_{i}$ such that for all $n>N_{i}$, $|f_{n}(\alpha_{i})-f(\alpha_{i})|<\epsilon/10$. Choose $N=\max(M,N_{0},\cdots,N_{M})$.

Consider an $x\in(\alpha_{i},\alpha_{i+1})$ for some $0\leq i<M$, and let $n>N$. Using uniform continuity of $f$, we have

[TABLE]

Further, we also have

[TABLE]

Convexity of $f_{n}$ implies

[TABLE]

Combining part of equation (82) and equation (83), we obtain

[TABLE]

We’ll now try to lower-bound $f_{n}(x)$. First consider the case when $i\geq 1$. In this case we have

[TABLE]

We write $\alpha_{i}$ as a linear combination of $x$ and $\alpha_{i-1}$, and use the convexity of $f_{n}$ to arrive at

[TABLE]

This implies

[TABLE]

Taking the infimum of the left side, we get

[TABLE]

Note that since the LHS is linear in $x$, the infimum occurs at one of the endpoints of the interval, $\alpha_{i}$ or $\alpha_{i+1}$. Substituting, we get

[TABLE]

Combining inequality (85) with a part of inequality (82), we have

[TABLE]

Combining (84) and (86) we conclude that for all $x\in(\alpha_{1},\alpha_{M})$, and for all $n>N$,

[TABLE]

Now let $x\in(\alpha_{0},\alpha_{1})$. We express $\alpha_{1}$ as a linear combination of $x$ and $\alpha_{2}$ and follows the steps as above to establish (86) for $x\in(\alpha_{0},\alpha_{1})$. This shows that for all $x\in[a,b]$, $||f_{n}(x)-f(x)||<\epsilon/2$ for all $n>N$, and concludes the proof. ∎

Appendix C Alternate definitions of typical sets

Theorem C.1**.**

Let $X$ be a non-uniform log-concave random variable. For $\epsilon>0$, define the following sequences of sets

[TABLE]

Let $\{T_{n}\}$ be any sequence of compact convex sets. Suppose that for any $\epsilon>0$ there exists an $N(\epsilon)$ such that for all $n>N(\epsilon)$ the following inclusion holds:

[TABLE]

Then the following equality holds for all $\theta\in[0,1]$:

[TABLE]

Remark C.1**.**

We exclude uniform random variables because $\underline{{\cal T}}^{\epsilon}_{n}=\phi$ for all $n$ and all $\epsilon>0$.

Proof.

By monotonicity of intrinsic volumes, we note that it is enough to show

[TABLE]

Using the same proof idea as in Lemma 2.1, we see that$\{\underline{{\cal T}}^{\epsilon}_{n}\}$ is a super-multiplicative sequence. Using the non-uniformity of $X$, we observe that for all small enough $\epsilon$ the set $\underline{{\cal T}}^{\epsilon}_{n}\neq\phi$ for all $n$. Furthermore, noting that $\underline{{\cal T}}^{\epsilon}_{n}\subseteq\overline{\cal T}^{\epsilon}_{n}$ we see that the intrinsic volumes of $\{\underline{{\cal T}}^{\epsilon}_{n}\}$ constitute a proper super-convolutive sequence. Thus, the limit function for the intrinsic volumes of $\{\underline{{\cal T}}^{\epsilon}_{n}\}$ exists, although it may not be continuous at 1 (continuity at 0 follows since it is bounded above $h_{X}$. For ease of notation we refer to the limit functions of the intrinsic volumes of $\{\underline{{\cal T}}^{\epsilon}_{n}\}$ and $\{\overline{\cal T}^{\epsilon}_{n}\}$ by $\underline{{\cal G}}^{\epsilon}$ and $\overline{\cal G}^{\epsilon}$. Our goal is to show that $\underline{{\cal G}}^{\epsilon}$ and $\overline{\cal G}^{\epsilon}$, cannot differ by too much. We show that it is possible to “bloat” $\{\underline{{\cal T}}^{\epsilon}_{n}\}$ by a small fraction, so that the bloated set will contain $\{\overline{\cal T}^{\epsilon}_{n}\}$.

Let $p_{X}(x)=\exp(-\Phi(x))$, and assume without loss of generality that $\Phi$ achieves its minimum at 0. Note that $h(X)=E\Phi(X)$, and thus $h(X)\geq\Phi(0)$. This inequality is strict when $X$ is not uniform. Assume $\epsilon<h(X)-\Phi(0)$. The two sequences of sets may be equivalently described as

[TABLE]

For $\alpha>0$, consider the map that takes $x^{n}\in\mathbb{R}$ to $(1+\alpha)x^{n}$. Let $x^{n}$ be a point on the boundary of $\underline{{\cal T}}^{\epsilon}_{n}$ satisfying $\sum\Phi(x_{i})=n(h(X)-\epsilon)$. We have the inequalities

[TABLE]

where $(a)$ follows from the convexity of $\Phi$. Now for a choice of $\alpha=\frac{2\epsilon}{h(X)-\epsilon-\Phi(0)}$, we will have

[TABLE]

that is, $x^{n}(1+\alpha)\notin\overline{\cal T}^{\epsilon}_{n}$. Note that $\alpha\to 0$ as $\epsilon\to 0$. Thus, for this choice of $\alpha$ we must have

[TABLE]

This implies

[TABLE]

We also have $\underline{{\cal G}}^{\epsilon}\leq\overline{\cal G}^{\epsilon}$, giving

[TABLE]

Since and $\lim_{\epsilon\to 0}\overline{\cal G}^{\epsilon}=h_{X}$, we can take the limit as $\epsilon\to 0$ to conclude that

[TABLE]

This concludes the proof. ∎

Appendix D Gärtner-Ellis theorem

Theorem D.1** (Gärtner-Ellis theorem).**

Consider a sequence of random vectors $Z_{n}\in\mathbb{R}^{d}$, where $Z_{n}$ possess the law $\mu_{n}$ and the logarithmic moment generating function

[TABLE]

We assume the following:

• $(*)$

For each $\lambda\in\mathbb{R}^{d}$, the limit

[TABLE]

exists as an extended a real number. Further the origin belongs to the interior ${\cal D}_{\Lambda}:=\{\lambda\in\mathbb{R}^{d}~{}|~{}\Lambda(\lambda)<\infty\}$.

Let $\Lambda^{*}$ be the convex conjugate of $\lambda$ with ${\cal D}_{\Lambda^{*}}=\{x\in\mathbb{R}^{d}~{}|~{}\Lambda^{*}(x)<\infty\}$. When assumption **()** holds, the following are satisfied:*

For any closed set $I$,

[TABLE]

• 2.

For any open set $F$,

[TABLE]

where ${\cal F}$ is the set of exposed points of $\Lambda^{*}$ whose exposing hyperplane belongs to the interior of ${\cal D}_{\Lambda}$.

• 3.

If $\Lambda$ is an essentially smooth, lower semicontinuous function, then the large deviations principle holds with a good rate function $\Lambda^{*}$.

Remark D.1**.**

For definitions of exposed points, essentially smooth functions and good rate function, we refer to Section $2.3$ of [13]. For our purpose, it is enough to know that if $\Lambda$ is differentiable on ${\cal D}_{\Lambda}=\mathbb{R}^{d}$, then is it essentially smooth.