The multidimensional truncated Moment Problem: Shape and Gaussian Mixture Reconstruction from Derivatives of Moments

TL;DR

This paper develops a theory for representing moment functionals using Gaussian mixtures and polytopes, establishing bounds on the number of Gaussians needed for such representations.

Contribution

It introduces derivatives of moments to analyze Gaussian mixture representations and determines minimal Gaussian counts for representing certain moment functionals.

Findings

Identifies exact Gaussian counts needed for specific moment functionals.

Establishes bounds on the number of Gaussians required for representation.

Provides theoretical limits for Gaussian mixture reconstruction from moments.

Abstract

In this paper we introduce the theory of derivatives of moments and (moment) functionals to represent moment functionals by Gaussian mixtures, characteristic functions of polytopes, and simple functions of polytopes. We study, among other measures, Gaussian mixtures, their reconstruction from moments and especially the number of Gaussians needed to represent moment functionals. We find that there are moment functionals $L:\mathbb{R}[x_1,\dots,x_n]_{\leq 2d}\to\mathbb{R}$ which can be represented by a sum of $\binom{n+2d}{n} - n\cdot \binom{n+d}{n} + \binom{n}{2}$ Gaussians but not less. Hence, for any $d\in\mathbb{N}$ and $\varepsilon>0$ we find an $n\in\mathbb{N}$ such that $L$ can be represented by a sum of $(1-\varepsilon)\cdot\binom{n+2d}{n}$ Gaussians but not less. An upper bound is $\binom{n+2d}{n}-1$.

Equations

$$s_{\alpha }:={\int }_{\text{mathds}{R}^n}{x}^{\alpha }\text{leavevmode}\mathrm{d}\mu (x)$$

$$s_a:={\int }_{\text{mathds}{R}^n}a(x)\text{leavevmode}\mathrm{d}\mu (x),$$

$$F(x):=\sum _{i=1}^k{c}_i\cdot \exp \left(-(x-b_i{)}^T{A}_i(x-b_i)\right)$$

$$\left(\genfrac{}{}{0ex}{}{n+2d}{n}\right)-n\cdot \left(\genfrac{}{}{0ex}{}{n+d}{n}\right)+\left(\genfrac{}{}{0ex}{}{n}{2}\right)$$

$$(1-\epsilon )\cdot \left(\genfrac{}{}{0ex}{}{n+2d}{n}\right)$$

$$L(a)={\int }_{\mathcal{X}}a(x)\text{leavevmode}\mathrm{d}\mu (x)\text{for all}a\in \mathcal{A},$$

$$s_i:=L(a_i)$$

$${\mathsf{A}}_{n,d}:\{x^{\alpha }|\alpha \in \text{mathds}{N}_0^n\wedge |\alpha |={\alpha }_1+\cdots +{\alpha }_m\le d\}$$

$$s_{\mathsf{A}}:\mathcal{X}\to \text{mathds}{R}^m,x\mapsto s_{\mathsf{A}}(x):=\left(\begin{array}{c}{a}_1(x)\\ ⋮\\ a_m(x)\end{array}\right).$$

$$s=\sum _{i=1}^k{c}_i\cdot s_{\mathsf{A}}(x_i)\text{resp.}L=\sum _{i=1}^k{c}_i\cdot l_{x_i}$$

$${\mathcal{C}}_{\mathsf{A}}(s):=\min \{k\in \text{mathds}{N}_0|\exists \mu \in {\mathfrak{M}}_{\mathsf{A}}(s)k\text{-atomic}\}.$$

$${\mathcal{C}}_{\mathsf{A}}:=\underset{s\in {\mathcal{S}}_{\mathsf{A}}}{\max }{\mathcal{C}}_{\mathsf{A}}(s).$$

$${\mathcal{C}}_{\mathsf{A}}\ge {\mathcal{C}}_{\mathsf{A}}(s)=\dim \mathrm{lin}\{s_{\mathsf{A}}(x_i)|i=1,\dots ,k\}.$$

$$(s,t,\dots,z)_{l}:=(s_{\alpha},t_{\alpha},\dots,z_{\alpha})_{\alpha\in\mathsf{N}:|\alpha|\leq{\color[rgb]{.75,0,.25}l}}$$

$${\int }_{\mathcal{X}}|a(x)|\text{leavevmode}\mathrm{d}{\delta }_{\sigma ,\xi }(x)<\infty .$$

$${\int }_{\mathcal{X}}a(x)\text{leavevmode}\mathrm{d}{\delta }_{\sigma ,\xi }\stackrel{\sigma \to {\sigma }_0}{\to }{\int }_{\mathcal{X}}a(x)\text{leavevmode}\mathrm{d}{\delta }_{\xi }(x)=a(\xi )$$

$$\sum _{i=1}^k{c}_i\cdot {\delta }_{{\sigma }_i,{\xi }_i}$$

$${\mathcal{C}}_{\mathsf{A}}^M(s):=\{k\in \text{mathds}{N}_0|s\text{has a mixture representation with}k\text{components}\}.$$

$${\mathcal{C}}_{\mathsf{A}}^M:=\underset{s\in {\mathcal{T}}_{\mathsf{A}}}{\max }{\mathcal{C}}_{\mathsf{A}}^M(s).$$

$$\mathrm{int}{\mathcal{T}}_{\mathsf{A}}=\mathrm{int}{\mathcal{S}}_{\mathsf{A}}.$$

$$s=\sum _{i=1}^k{c}_i\cdot s_{{\mathsf{A}}_{1,2d+1}}(z_i)$$

$${\mathcal{H}}_d(M_{1}s)v_i=z_i{\mathcal{H}}_d(s)v_i.$$

$${\mathcal{H}}_d(s)=(s_{{\mathsf{A}}_{1,d}}(z_1),\dots ,s_{{\mathsf{A}}_{1,d}}(z_k))\cdot \mathrm{diag}(c_1,\dots ,c_k)\cdot (s_{{\mathsf{A}}_{1,d}}(z_1),\dots ,s_{{\mathsf{A}}_{1,d}}(z_k){)}^T$$

$$\begin{gathered} {\mathcal{H}}_d(M_{1}s)= \\ (s_{{\mathsf{A}}_{1,d}}(z_1),\dots ,s_{{\mathsf{A}}_{1,d}}(z_k))\cdot \mathrm{diag}(c_1{z}_1,\dots ,c_k{z}_k)\cdot (s_{{\mathsf{A}}_{1,d}}(z_1),\dots ,s_{{\mathsf{A}}_{1,d}}(z_k){)}^T.\text{qed} \end{gathered}$$

$$\{x_1,\dots ,x_k\}=\mathcal{Z}(p)\text{and}p(x)=v_k+v_{k-1}x+\cdots +v_0{x}^k.$$

$$\mu ({\partial }_{x}f)$$

$$=:-{\int }_{\text{mathds}R}f\text{leavevmode}\mathrm{d}({\partial }_x\mu )=-({\partial }_x\mu )(f)$$

$${\partial }^{\alpha }{s}_i:=(-1{)}^{|\alpha |}\cdot L({\partial }^{\alpha }{a}_i).$$

$${\partial }^{\alpha }s:=({\partial }^{\alpha }{s}_1,\dots ,{\partial }^{\alpha }{s}_m)$$

$$({\partial }^{\alpha }L)(a):=(-1{)}^{|\alpha |}\cdot L({\partial }^{\alpha }a)$$

$${\partial }^{\beta }s=({\partial }^{\beta }{s}_{\alpha })\text{with}{\partial }^{\beta }{s}_{\alpha }=(-1{)}^{|\beta |}\mu ({\partial }^{\beta }{x}^{\alpha })=\{\begin{array}{ll}(-1{)}^{|\beta |}\cdot \frac{\alpha !}{(\alpha -\beta )!}{s}_{\alpha -\beta }& \text{if}\beta \le \alpha ,\\ 0& \text{else}\end{array},$$

Full text

AMS Subject Classification (2010). 44A60, 14P99, 30E05, 65D32, 35R30.

Key words: truncated moment problem, Carathéodory number, measure reconstruction, Gaussian mixture, generalized eigenvalues, shape reconstruction, algebraic statistics, integral representation

1. Introduction

Reconstructing measures from moments is a key problem in statistics [Pea94, TSM85, MMR05, dD19], shape reconstruction [Bal61, MN68, MR80, LR82, MVKW95, GMV99, BGL07, GLPR12, GNPR14, GPSS18, KSS18], pattern recognition [Hu62, DBN92, Che93, SMD*+*07, APST19], financial mathemiatics [Ana06, Sto16], and many other fields, and attracts increasing attention especially with the growing usage of computer programs and algorithms to handle such problems. But despite of its growing importance and wide range of application, the theoretical knowledge on the problem of reconstructing measures from moments is very small, especially when only finitely many moments are known. For instance, only recently [dD19] the question of which truncated moment sequences are represented by Gaussian, log-normal, and more general mixtures was fully answered and the first non-trivial bounds on the required number of summands were given.

While derivatives in the context of moments have been used before, surprisingly no unified approach was introduced so far. In the present paper we present the first unified and systematic approach to reconstruct and investigate measures from moments: derivatives of moments. In Section 4 we define and investigate derivatives of moments and show that the derivative $\partial^{\alpha}L:=(-1)^{|\alpha|}\cdot L\circ\partial^{\alpha}$ of a (moment) functional $L$ is represented by the distributional derivative $\partial^{\alpha}\mu$ of a representing measure $\mu$ of $L$. From this treatment it is clear that $\partial^{\alpha}L$ is an object that is interesting to investigate on its own account and not only because it solves problems and appears (implicitly or explicitly) in proofs and calculations.

In Section 5 we use the concept of derivatives of moments to reprove several known results on reconstructing polytopes and special measures in a unified and efficient way. Proofs formerly presented over several pages now reduce to a few lines and their key arguments become much more apparent. We use these simplified arguments and proofs to extend these results, e.g., we extend the results from polynomial moments

[TABLE]

with $x^{\alpha}=x_{1}^{\alpha_{1}}\cdots x_{n}^{\alpha_{n}}$ to non-polynomial moments:

[TABLE]

where $a$ is a measurable (differentiable) function. This allows us to formulate results in full generality and $\partial^{\alpha}L$ can still be easily calculated from $L$.

In Section 6 we return to the reconstruction and investigation of (Gaussian) mixtures. Based on derivatives of moments we fully characterize moment sequences from one ($n$-dimensional) Gaussian distribution $c\cdot\exp\left(-(x-b)^{T}A(x-b)\right)$ and we determine $b\in\mathds{R}^{n}$ and $A\in\mathds{R}^{n\times n}$ from the moments. While this was known before, our simplified arguments and proofs using derivatives of moments enable us to extend this to mixtures, i.e., linear combinations of e.g. Gaussian distributions:

[TABLE]

with $c_{i}\in\mathds{R}$ ($c_{i}>0$), $b_{i}\in\mathds{R}^{n}$ and $A_{i}\in\mathds{R}^{n\times n}$ for all $i=1,\dots,k$. In the one-dimensional case ($n=1$) we give an explicit way to determine the parameters in (1). Simple formulas are gained under the restriction that $A_{1},\dots A_{k}\in\mathds{R}$ are all equal: $A_{1}=\dots=A_{k}$. But before we allow the possible relaxation to arbitrary $A_{1},\dots,A_{k}\in\mathds{R}^{n\times n}$ we examine the number $k$ of mixtures required to represent a moment sequence $s$, i.e., its minimal number, the (mixture) Carathéodory number $\mathcal{C}_{\mathsf{A}}^{M}(s)$. Based on very recent results on the Carathéodory number $\mathcal{C}_{\mathsf{A}}$ (number of Dirac delta measures, i.e., point evaluations) in [RS18, dDS18a, dDS18b] and especially [dDK19] we derive new lower bounds and asymptotic limits for the case of mixtures as well. We show that a non-zero (polynomial) function $p$ with finitely many zeros $\mathcal{Z}(p)$ gives a moment sequence $s$, resp. moment functional $L$, which needs as many components in a mixtures representation as there are linearly independent point evaluation located at $\mathcal{Z}(p)$, see 6.22. As a consequence (6.26) we find that there are moment functionals $L:\mathds{R}[x_{1},\dots,x_{n}]_{\leq 2d}\to\mathds{R}$, which can be represented by a sum of

[TABLE]

Gaussian distributions but not less. This disproves the belief that allowing arbitrary $A_{i}\in\mathds{R}^{n\times n}$ with $A_{i}\succ 0$ reduces the number $k$ of components. Finally, (2) shows that for each $d\in\mathds{N}$ and $\varepsilon>0$ there is an $n\in\mathds{N}$ and a moment functional $L:\mathds{R}[x_{1},\dots,x_{n}]_{\leq 2d}\to\mathds{R}$ which can be represented by a sum of

[TABLE]

Gaussian distributions but not less.

2. Preliminaries

Let $\mathcal{A}$ be a (finite dimensional) real vector space of measurable functions on a measurable space $(\mathcal{X},\mathfrak{A})$. Denote by $L:\mathcal{A}\rightarrow\mathds{R}$ a continuous linear functional. If there is a (positive) measure $\mu$ on $(\mathcal{X},\mathfrak{A})$ such that

[TABLE]

then $L$ is called a moment functional. If $\mathcal{A}$ is finite dimensional, it is a truncated moment functional. By $\mathsf{A}=\{a_{1},\dots,a_{m}\}$ we denote a basis of the $m$-dimensional real vector space $\mathcal{A}$ and by

[TABLE]

the $a_{i}$-th (or simply $i$-th) moment of $L$ (or $\mu$ for a $\mu$ as in (3)). Given a sequence $s=(s_{1},\dots,s_{m})\in\mathds{R}^{m}$ we define the Riesz functional $L_{s}$ by setting $L_{s}(a_{i})=s_{i}$ for all $i=1,\dots,m$ and extending it linearly to $\mathcal{A}$, i.e., the Riesz functional induces a bijection between moment sequences $s=(s_{1},\dots,s_{m})$ and moment functionals $L=L_{s}$. By $\mathfrak{M}_{\mathsf{A}}$ we denote the set of all measures on $(\mathcal{X},\mathfrak{A})$ such that all $a\in\mathcal{A}$ are integrable and by $\mathfrak{M}_{\mathsf{A}}(s)$ or $\mathfrak{M}_{\mathsf{A}}(L)$ we denote all representing measures of the moment sequence $s$ resp. moment functional $L$. Since the polynomials $\mathds{R}[x_{1},\dots,x_{n}]_{\leq 2d}$ are of special importance, we denote by

[TABLE]

the monomial basis, where we have $x^{\alpha}=x_{1}^{\alpha_{1}}\cdots x_{n}^{\alpha_{n}}$ with $\alpha=(\alpha_{1},\dots,\alpha_{n})\in\mathds{N}_{0}^{n}$. On $\mathds{N}_{0}^{n}$ we work with the partial order $\alpha=(\alpha_{1},\dots,\alpha_{n})\leq\beta=(\beta_{1},\dots,\beta_{n})$ if $\alpha_{i}\leq\beta_{i}$ for all $i=1,\dots,n$.

Definition 2.1**.**

Let $\mathsf{A}=\{a_{1},\dots,a_{m}\}$ be a basis of the finite dimensional vector space $\mathcal{A}$ of measurable functions on the measurable space $(\mathcal{X},\mathfrak{A})$. We define $s_{\mathsf{A}}$ by

[TABLE]

Of course, $s_{\mathsf{A}}(x)$ is the moment sequence of the Dirac $\delta_{x}$ measure and the corresponding moment functional is the point evaluation $l_{x}$ with $l_{x}(a):=a(x)$. By a measure we always mean a positive measure unless it is explicitly denoted as a signed measure.

The fundamental theorem in the theory of truncated moments is the following.

Theorem 2.2** (Richter Theorem [Ric57]).**

Let $\mathsf{A}=\{a_{1},\dots,a_{m}\}$, $m\in\mathds{N}$, be finitely many measurable functions on a measurable space $(\mathcal{X},\mathfrak{A})$. Then every moment sequence $s\in\mathcal{S}_{\mathsf{A}}$ resp. moment functional $L:\mathcal{A}\to\mathds{R}$ has a $k$-atomic representing measure

[TABLE]

with $k\leq m$, $c_{1},\dots,c_{k}>0$, and $x_{1},\dots,x_{k}\in\mathcal{X}$.

The theorem can also be called Richter–Rogosinski–Rosenbloom Theorem [Ric57, Rog58, Ros52], see the discussion after Example 20 in [dDS18a] for more details. That every truncated moment sequence has a $k$-atomic representing measure ensures that the Carathéodory number $\mathcal{C}_{\mathsf{A}}$ is well-defined.

Definition 2.3**.**

Let $\mathsf{A}=\{a_{1},\dots,a_{m}\}$ be linearly independent measurable functions on a measurable space $(\mathcal{X},\mathfrak{A})$. For $s\in\mathcal{S}_{\mathsf{A}}$ we define the Carathéodory number $\mathcal{C}_{\mathsf{A}}(s)$ of $s$ by

[TABLE]

We define the Carathéodory number $\mathcal{C}_{\mathsf{A}}$ of $\mathcal{S}_{\mathsf{A}}$ by

[TABLE]

The same definition holds for moment functionals $L:\mathcal{A}\rightarrow\mathds{R}$.

The following theorem turns out to be a convenient tool for proving lower bounds on the Carathéodory number $\mathcal{C}_{\mathsf{A}}$.

Theorem 2.4** ([dDS18b, Thm. 18]).**

Let $\mathsf{A}=\{a_{1},\dots,a_{m}\}$ be measurable functions on a measurable space $(\mathcal{X},\mathfrak{A})$, $s\in\mathcal{S}_{\mathsf{A}}$, and $a\in\mathcal{A}$ with $a\geq 0$ on $\mathcal{X}$, $\mathcal{Z}(a)=\{x_{1},\dots,x_{k}\}$ and $L_{s}(a)=0$. Then

[TABLE]

Remark 2.5**.**

Note that in 2.4 it is crucial that the zero set of $a$ is finite: Take $a=0$ and $\mathcal{X}=\mathds{R}^{n}$ for a simple example where the statement fails when the zero set is not finite.

It is well-known that in general not every sequence $s\in\mathds{R}^{m}$ or linear functional $L:\mathcal{A}\rightarrow\mathds{R}$ has a positive representing measure. But of course it always has a signed $k$-atomic representing measure with $k\leq m$.

Lemma 2.6** ([dDS18a, Prop. 12]).**

Let $\mathsf{A}=\{a_{1},\dots,a_{m}\}$ be a basis of the finite dimensional space $\mathcal{A}$ of measurable functions on a measurable space $(\mathcal{X},\mathfrak{A})$. There exist points $x_{1},\dots,x_{m}\in\mathcal{X}$ such that every vector $s\in\mathds{R}^{m}$ has a signed $k$-atomic representing measure $\mu$ with $k\leq m$ and all atoms are from $\{x_{1},\dots,x_{m}\}$, i.e., every functional $L:\mathcal{A}\rightarrow\mathds{R}$ is the linear combination $L=c_{1}l_{x_{1}}+\cdots+c_{m}l_{x_{m}}$, $c_{i}\in\mathds{R}$.

It is well-known that in dimension $n=1$ the atom positions $x_{i}$ of a moment sequence can be calculated from the generalized eigenvalue problem, see e.g. [GMV99]. To formulate this and other results we introduce the following shift.

Definition 2.7**.**

Let $n,d\in\mathds{N}$ and $s=(s_{\alpha})_{\alpha\in\mathds{N}_{0}^{n}:|\alpha|\leq d}$. For $\beta\in\mathds{N}_{0}^{n}$ with $|\beta|\leq d$ we define $M_{\beta}s:=(M_{\beta}s_{\alpha})_{\alpha\in\mathds{N}_{0}^{n}:|\alpha+\beta|\leq d}$ by $M_{\beta}s_{\alpha}:=s_{\alpha+\beta}$, i.e., $(M_{\beta}L)(p)=L(x^{\beta}\cdot p)$.

For a space $\mathcal{A}$ of measurable functions with basis $\mathsf{A}=\{a_{1},a_{2}\dots\}$ the Hankel matrix $\mathcal{H}_{d}(L)$ of a linear functional $L:\mathcal{A}^{2}\rightarrow\mathds{R}$ is given by $\mathcal{H}_{d}(L)=(L(a_{i}a_{j}))_{i,j=1}^{d}$. The atom positions of a truncated moment sequence $s$ (resp. moment functional $L$) are then determined from results in Section 3.

We use the following notation.

Definition 2.8**.**

Let $s=(s_{\alpha})_{\alpha\in\mathsf{N}},t=(t_{\alpha})_{\alpha\in\mathsf{N}},\dots,z=(z_{\alpha})_{\alpha\in\mathsf{N}}$ be multi-indexed sequences $\alpha\in\mathsf{N}\subseteq\mathds{N}_{0}^{n}$ and $l\in\mathds{N}$. We define the matrix

[TABLE]

For (Gaussian) mixtures we use the following general setting as in [dD19]:

Definition 2.9**.**

Let $\Sigma$ be some fixed set of parameters (in a larger metric space). For all $\sigma\in\Sigma$ and $\xi\in\mathcal{X}$ we let $\delta_{\sigma,\xi}$ denote probability measures on the measurable space $(\mathcal{X},\mathfrak{A})$ such that:

1. i)

All $a\in\mathcal{A}$ are $\delta_{\sigma,\xi}$-measurable for all $(\sigma,\xi)\in\Sigma\times\mathcal{X}$, i.e.,

[TABLE] 2. ii)

There exists a (unique) $\sigma_{0}\in\overline{\Sigma}$ (closure of $\Sigma$) such that

[TABLE]

for all $a\in\mathcal{A}$ and $\xi\in\mathcal{X}$.

For $k\in\mathds{N}$ and $\sigma_{1},\dots,\sigma_{k}\in\Sigma$, and $\xi_{1},\dots,\xi_{k}\in\mathcal{X}$ a ($k$-)mixture is then

[TABLE]

where $\delta_{\sigma_{i},\xi_{i}}$ is its $i$-th component. We have $c_{i}\geq 0$ unless we explicitly speak of signed mixtures ($c_{i}\in\mathds{R}$).

Examples $(\delta_{\sigma,\xi},\mathcal{X},\Sigma,\sigma_{0})$ of this general setting are Gaussian and log-normal measures, see [dD19]. There we already treated the Carathéodory number $\mathcal{C}_{\mathsf{A}}^{M}$ of mixtures and answered which moment sequences can be represented by mixtures.

Definition 2.10**.**

If $s\in\mathcal{S}_{\mathsf{A}}$ has a mixture representation, then we define its (mixture) Carathéodory number $\mathcal{C}^{M}_{\mathsf{A}}(s)$ by

[TABLE]

We call $\mathcal{T}_{\mathsf{A}}\subseteq\mathcal{S}_{\mathsf{A}}$ the mixture cone, i.e., the set of all moment sequences which have a (finite) mixture representation. The (mixture) Carathéodory number $\mathcal{C}_{\mathsf{A}}^{M}$ is then defined by

[TABLE]

Of course, since we always have $\mathcal{C}_{\mathsf{A}}^{M}\leq\dim\mathcal{A}$, $\mathcal{C}_{\mathsf{A}}^{M}$ is well-defined. In [dD19] we gave upper bounds on $\mathcal{C}_{\mathsf{A}}^{M}$.

Theorem 2.11** ([dD19, Thm. 17(ii)]).**

Let $\mathcal{A}$ be a finite-dimensional space of continuous functions and $\delta_{\sigma,\xi}$ probability measures as in 2.9. Then

[TABLE]

More on the (truncated) moment problem can be found e.g. in [Sti94, ST43, Akh65, KN77, Kem68, Kem87, Lan80, Mar08, Lau09, FN10, Las15, Sch17] and references therein.

3. Reconstruction of atomic Measures

For one-dimensional moment sequences the atom positions of an atomic representing measure can be determined by the following to results.

Lemma 3.1**.**

Let $n,d\in\mathds{N}$, $\mathcal{X}=\mathds{C}^{n}$, and $s=(s_{0},s_{1},\dots,s_{2d+1})\in\mathds{R}^{2d+2}$ with

[TABLE]

for some $z_{i}\in\mathds{C}$, $c_{i}\in\mathds{C}$, and $k\leq d$. Then the $z_{i}$ are unique and are the eigenvalues of the generalized eigenvalue problem

[TABLE]

Proof.

That the $z_{i}$ are the eigenvalues of (4) and therefore uniqueness follows from

[TABLE]

and

[TABLE]

Lemma 3.2**.**

Let $s=(s_{0},\dots,s_{2k})\in\mathds{R}^{2k+1}$ be a sequence with $\mathcal{H}(s)\succeq 0$ and $\mathcal{H}(s)$ is singular (with kernel dimension one). Let $v=(v_{k},\dots,v_{1},v_{0})$ with $\ker\mathcal{H}(s)=v\cdot\mathds{R}$. Then $s$ has a $k$-atomic representing measure $\mu=\sum_{i=1}^{k}c_{i}\cdot\delta_{x_{i}}$ with

[TABLE]

Proof.

$L_{s}(p^{2})=v^{T}\mathcal{H}(s)v=0$, i.e., $\mathrm{supp}\,\mu\subseteq\mathcal{Z}(p^{2})=\mathcal{Z}(p)$. Equality holds since we work in the one-dimensional framework: $k=|\mathrm{supp}\,\mu|=\mathrm{rank}\,\mathcal{H}(s)=|\mathcal{Z}(p)|$. ∎

Compare the preceding results with Vieta’s Formulas (6.6).

4. Derivatives of Moments and Measures

The following simple and well-known example from the theory of distributions is our motivation in this section. As in the theory of distributions we denote by $\mathcal{D}(\Omega)=C_{0}^{\infty}(\Omega,\mathds{R})$ the set of all test functions and by $\mathcal{D}^{\prime}(\Omega)$ the set of all distributions (continuous linear functionals on $\mathcal{D}(\Omega)$). Most of our applications and examples will work on $\Omega=\mathcal{X}=\mathds{R}^{n}$, $n\in\mathds{N}$.

Example 4.1**.**

Let $\mu$ on $\mathcal{X}=\mathds{R}$ be given by $\mathrm{d}\mu:=\chi_{[a,b]}\cdot\mathrm{d}\lambda$, where $\chi_{[a,b]}$ is the characteristic function of the set $[a,b]$, $a<b$, and $\lambda$ is the Lebesgue measure on $\mathds{R}$. For $f\in C^{1}(\mathds{R},\mathds{R})$ we have

[TABLE]

where we understand $\partial_{x}\chi_{[a,b]}$ in the distributional sense [Gru09] and $\partial_{x}\mu=\delta_{a}-\delta_{b}$ as defined above.

Derivatives of Moments

Distribution theory motivates the following definition.

Definition 4.2**.**

Let $\mathcal{A}$ be a (finite dimensional) vector space of measurable functions, $L:\mathcal{A}\rightarrow\mathds{R}$ be a linear functional, and $\alpha\in\mathds{N}_{0}^{n}$. If $\partial^{\alpha}a_{i}\in\mathcal{A}$ for some $a_{i}\in\mathcal{A}$ we define the $\alpha$-th derivative $\partial^{\alpha}s_{i}$ of $s_{i}=L(a_{i})$ by

[TABLE]

Let $\mathsf{A}=\{a_{1},\dots,a_{m}\}$, $m\in\mathds{N}\cup\{\infty\}$, be a basis of $\mathcal{A}$. If $\partial^{\alpha}\mathcal{A}\subseteq\mathcal{A}$, then we define the $\alpha$-th derivative of the sequence $s=(L(a_{1}),\dots,L(a_{m}))$ by

[TABLE]

or equivalently $\partial^{\alpha}L$ is defined by

[TABLE]

for all $a\in\mathcal{A}$ with $\mathcal{A}$ finite or infinite dimensional.

Since $\partial^{\alpha}\mathcal{A}\subseteq\mathcal{A}$ we can calculate $\partial^{\alpha}s_{i}$ directly from $L$.

Lemma 4.3**.**

If $\partial^{\alpha}a_{i}=\sum_{j=1}^{k}c_{j}a_{j}\in{\mathcal{A}}$, $k\in\mathds{N}$, then $\partial^{\alpha}s_{i}=(-1)^{|\alpha|}\cdot\sum_{j=1}^{k}c_{j}s_{j}$.

Proof.

$\partial^{\alpha}s_{i}=(-1)^{|\alpha|}\cdot{L}(\partial^{\alpha}a_{i})=(-1)^{|\alpha|}\cdot\sum_{j=1}^{k}c_{j}{L}(a_{j})=(-1)^{|\alpha|}\cdot\sum_{j=1}^{k}c_{j}s_{j}$. ∎

This provides us with explicit ways to calculate $\partial^{\alpha}s$ as the next examples show.

Example 4.4**.**

1. a)

Let $\mathsf{A}=\mathsf{A}_{n,d}$ on $\mathds{R}^{n}$, $s=(s_{\alpha})\in\mathcal{S}_{\mathsf{A}_{n,d}}$, and $\beta\in\mathds{N}_{0}^{n}$. We have

[TABLE]

see also (12) in 4.17 for $n=1$. 2. b)

Let $\mathsf{A}=\{\exp(d_{1}x),\dots,\exp(d_{m}x)\}$ on $\mathds{R}$ with $d_{1}<\dots<d_{m}$ and $s=(s_{i})_{i=1}^{m}\in\mathcal{S}_{\mathsf{A}}$. Then $\partial_{x}^{k}s=(d_{1}^{k}\cdot s_{1},\dots,d_{m}^{k}\cdot s_{m})$. 3. c)

Let $\mathsf{A}=\{\sin x,\cos x,\dots,\sin(kx),\cos(kx)\}$ on $\mathds{R}$ (or $[0,2\pi)$) for a $k\in\mathds{N}$ and $s=(s_{1},s_{2},\dots,s_{2k-1},s_{2k})\in\mathcal{S}_{\mathsf{A}}$. Then

[TABLE]

Note that $\partial^{\alpha}$ and $M_{\beta}$ in 2.7 “almost” commute.

Lemma 4.5**.**

For $\mathsf{A}=\mathsf{A}_{n,d}$, $s=(s_{\gamma})\in\mathds{R}^{|\mathsf{A}_{n,d}|}$ we have

[TABLE]

Remark 4.6**.**

When $s$ resp. $L$ is a moment sequence/functional, then $\partial s$ resp. $\partial L$ (or $-\partial s$ resp. $-\partial L$) is in general not a moment sequence. Let $s=(1,1,1)$ be the moment sequence of $\delta_{1}$ with $\mathsf{A}=\{1,x,x^{2}\}$, then $\partial s=(0,-1,-2)$, i.e., $(\partial L)(1)=L_{\partial s}(1)=0$ but $(\partial L)(x)=L_{\partial s}(x)\neq 0$.

Lemma 4.7**.**

Let $\mathcal{A}$ be a vector space of measurable functions on the measurable space $(\mathcal{X},\mathfrak{A})$, $1\in\mathcal{A}$, $\alpha\in\mathds{N}_{0}^{n}$, $\alpha\neq 0$, $\partial^{\alpha}\mathcal{A}\subseteq\mathcal{A}$, and $L:\mathcal{A}\rightarrow\mathds{R}$ a linear functional. The following are equivalent:

1. i)

$\partial^{\alpha}L$* is a moment functional.* 2. ii)

$\partial^{\alpha}L=0$.

Proof.

While (ii) $\Rightarrow$ (i) is clear, for (i) $\Rightarrow$ (ii) let $\mu$ be a representing measure of $\partial^{\alpha}L$. Then

[TABLE]

i.e., $\mu=0$ and therefore $\partial^{\alpha}L=0$. ∎

Remark 4.8**.**

Let $\mathcal{A}$ be a vector space of measurable functions on the measurable space $(\mathcal{X},\mathfrak{A})$, $\alpha\in\mathds{N}_{0}^{n}$, $\alpha\neq 0$ such that $\partial^{\alpha}\mathcal{A}\subseteq\mathcal{A}$. The following are equivalent:

1. i)

For every linear functional $L:\mathcal{A}\rightarrow\mathds{R}$ there exists a $K:\mathcal{A}\rightarrow\mathds{R}$ with $\partial^{\alpha}K=L$. 2. ii)

$\partial^{\alpha}:\mathcal{A}\rightarrow\mathcal{A}$ is injective.

Indeed, $\partial^{\alpha}:\mathcal{A}\rightarrow\mathcal{A}$ is injective, if and only if the induced endomorphism

[TABLE]

of the dual space is surjective. In 4.4 (a) $\partial^{\alpha}$ ($\alpha\neq 0$) is not injective, in (c) $\partial^{k}$ is injective, and in (b) $\partial^{k}$ is injective if and only if $d_{i}\neq 0$ for all $i=1,\dots,m$.

Derivatives of Measures

In 4.1 we have seen that for the specific measure $\mu$ with $\mathrm{d}\mu=\chi_{[a,b]}\leavevmode\nobreak\ \mathrm{d}\lambda$ the derivative is $\partial_{x}\mu=\delta_{a}-\delta_{b}$, of course in the distributional sense:

[TABLE]

Here we make use of the notation $\mu(f)$ for $\int f\leavevmode\nobreak\ \mathrm{d}\mu$ from the theory of distributions that comes in very handy. Note that we can even choose $f\in C^{\infty}(\mathds{R},\mathds{R})$ since $\mathrm{supp}\,\mu$ is compact and therefore compactness of $\mathrm{supp}\,f$ can be omitted. For the rest of this section we want to define $\partial^{\alpha}\mu$ for measures $\mu$, especially $\mu\in\mathfrak{M}_{\mathsf{A}}(s)$, if it exists.

Definition 4.9**.**

Let $\mathcal{A}$ be a (finite dimensional) vector space of measurable functions, $\mu$ a (signed) measure and $\alpha\in\mathds{N}_{0}^{n}$. Assume that $\partial^{\alpha}\mathcal{A}\subseteq\mathcal{A}$ and there exists a $\nu\in\mathcal{D}^{\prime}(\mathcal{X})$ such that

[TABLE]

If $\nu$ is a (signed) measure such that all $a\in\mathcal{A}$ are $\nu$-integrable, then we say the $\alpha$-th derivative $\partial^{\alpha}\mu$ of $\mu$ exists on $\mathcal{A}$ and is defined by

[TABLE]

The following statement, which connects 4.2 with 4.9, is the crucial observation of this section. It enables us to apply results from the theory of distributions to derivatives of moment functionals.

Theorem 4.10**.**

Let $\mathcal{A}$ be a (finite or infinite dimensional) vector space of measurable functions on the measurable space $(\mathcal{X},\mathfrak{A})$, $L:\mathcal{A}\rightarrow\mathds{R}$ be a moment functional with representing measure $\mu$, and $\alpha\in\mathds{N}_{0}^{n}$ such that $\partial^{\alpha}\mathcal{A}\subseteq\mathcal{A}$. If $\partial^{\alpha}\mu$ exists on $\mathcal{A}$, then $\partial^{\alpha}\mu$ is a (signed) representing measure of $\partial^{\alpha}L$, i.e.,

[TABLE]

Proof.

Since $\partial^{\alpha}\mu(a)$ exists for all $a\in\mathcal{A}$ we have

[TABLE]

Remark 4.11**.**

4.10 says that we can compute the derivative of a moment functional $L$ on $\mathcal{A}$ by taking the derivative of a representing measure $\mu$ (if its derivative exists on $\mathcal{A}$) and vice versa. In particular, the result does not depend on the choice of the representing measure.

Example 4.12**.**

Let $x\in\mathcal{X}\subset\mathds{R}^{n}$, $\alpha\in\mathds{N}_{0}^{n}$, $\mathcal{A}\subset C^{|\alpha|}(\mathcal{X},\mathds{R})$, and $\mu=\delta_{x}$, then $\partial^{\alpha}\mu=\partial^{\alpha}\delta_{x}$ is given by

[TABLE]

Hence, $\delta_{x}$ is an example of a measure whose derivative is no longer a measure.

Besides the Dirac measures also measures of the form $f\leavevmode\nobreak\ \mathrm{d}\lambda^{n}$ are very important, where $\lambda^{n}$ is the $n$-dimensional Lebesgue measure and $f$ is a measurable function.

Definition 4.13** ([Gru09, Eq. (3.2)]).**

Let $f\in L^{1}_{\mathrm{loc}}(\mathcal{X})$ and $\lambda^{n}$ the $n$-dimensional Lebesgue measure on $\mathcal{X}$. We define the distribution $\Lambda_{f}$ by

[TABLE]

Theorem 4.14** ([Gru09, Eqs. (3.15) and (3.21)]).**

Let $\alpha\in\mathds{N}_{0}^{n}$. Then

[TABLE]

If $\partial^{\alpha}\Lambda_{f}$ exists on $\mathcal{A}$, then by 4.10 we have

[TABLE]

The following example will be most important in the reconstruction of polytopes and simple functions from their moments, see Section 5.

Example 4.15**.**

Let $f:\mathds{R}\rightarrow\mathds{R}$ be a continuous and piece-wise linear function with compact support. Let $\xi_{1}<\dots<\xi_{k}$ be the points where $f$ is not differentiable. Then $(\Lambda_{f})^{\prime}=\sum_{i=1}^{k-1}c_{i}\cdot\chi_{(\xi_{i},\xi_{i+1}]}$ and $(\Lambda_{f})^{\prime\prime}=\sum_{i=1}^{k}(c_{i}-c_{i-1})\cdot\delta_{\xi_{i}}$ where $c_{i}=\frac{f(\xi_{i+1})-f(\xi_{i})}{\xi_{i+1}-\xi_{i}}$ for $i=1,\ldots,k-1$ and $c_{0}=c_{k}=0$ are the slopes of $f$. In particular, $(\Lambda_{f})^{\prime\prime}$ is a signed $k$-atomic measure.

Example 4.16**.**

Let $p_{i,j}\in\mathds{R}$ be points, $i=1,\dots,n$ and $j=0,1$. We define the $n$-dimensional hyperrectangle $\square_{p}$ of $p=(p_{1,0},p_{1,1},\dots,p_{n,0},p_{n,1})\in\mathds{R}^{2n}$ by

[TABLE]

The vertices of $\square_{p}$ are $p_{J}=(p_{1,j_{1}},\dots,p_{n,j_{n}})$ for all $J=(j_{1},\dots,j_{n})\in\{0,1\}^{n}$. Since $\square_{p}$ is compact all moments

[TABLE]

for $\alpha\in\mathds{N}^{n}_{0}$ exist. Here we abbreviated the characteristic function $\chi_{\square_{p}}$ of $\square_{p}$ as $\chi_{p}$. Set $\mathds{1}:=(1,\dots,1)$. From the Definitions 4.2 and 4.9 as well as 4.14 we find that

[TABLE]

has the signed representing measure

[TABLE]

supported only at the vertices $p_{J}$ of $\square_{p}$ where $|J|=j_{1}+\dots+j_{n}$.

Gaussian distributions will be considered in Section 6.

Example 4.17**.**

For $\Lambda_{f}$ with $f(x)=\exp(-|ax-b|^{k})$ all $i$-th moments

[TABLE]

exist where $a\in\mathds{R}_{>0}$, $b\in\mathds{R}$, and $k>0$. For $l\in\mathds{N}_{0}$ we find from the Definitions 4.2 and 4.9 as well as 4.14 that

[TABLE]

has a signed representing measure given by

[TABLE]

for suitable polynomials $h_{l}$. For $k=2$ we have $h_{l}(x)=(-a)^{l}\cdot H_{l}(ax-b)$ where $H_{l}$ is the $l$-th Hermite polynomial:

[TABLE]

5. Applications

Polytope Reconstruction

The problem of reconstructing a (convex and full-dimensional) polytope $P\subset\mathds{R}^{n}$, i.e., finding all vertices, is an extensively studied question and several algorithms have been proposed, see e.g. [Bal61, MN68, MR80, LR82, MVKW95, GMV99, BGL07, GLPR12, GNPR14, GPSS18, KSS18], and references therein.

Based on derivatives of moments we will present a simple proof of one version of these algorithms which calculates the vertices from finitely many moments

[TABLE]

We use the Brion–Lawrence–Khovanskii–Pukhlikov–Barvinok (BBaKLP) formulas [Bri88, Law91, Bar91, PK92, Bar92] and the generalized eigenvalue problem (as in 3.1). The aim is to convince the reader that derivatives of moments are a convenient tool for proving and extending the statement in a concise and conceptual way.

Let us state the BBaKLP formulas. This presentation is taken from [GLPR12]. Let $P$ be a polytope in $\mathds{R}^{n}$ with vertices $v_{1},\dots,v_{k}$ ($k\geq n+1$), then

[TABLE]

see [GLPR12, Eq. (3)], and for $j=n,n+1,\dots$ we have

[TABLE]

see [GLPR12, Eq. (4)], where $\tilde{D}_{v_{i}}(r)$ is a rational function on $r\in\mathds{R}^{n}$, i.e., $r$ can be chosen in general position such that $\tilde{D}_{v_{i}}(\,\cdot\,)$ has no zero or pole at $r$. The $s_{j}(r)$ is the $j$-th directional moment with direction $r$.

Definition 5.1**.**

Let $k,n\in\mathds{N}$, $P$ be a polytope with vertices $v_{1},\dots,v_{k}\in\mathds{R}^{n}$, $r\in\mathds{R}^{n}\setminus\{0\}$ a vector (of length 1), $a\in\mathds{R}$, and $H_{r,a}:=\{x\in\mathds{R}^{n}\,|\,\langle r,x\rangle=a\}$ be an affine hyperplane with normal vector $r$. We define the area function $\Theta_{P,r}$ to be the $(n-1)$-dimensional volume of $P\cap H_{r,x}$

[TABLE]

where $\lambda^{n-1}$ is the $(n-1)$-dimensional Lebesgue measure on $H_{r,x}$.

Of course, the area function is integration by parts

[TABLE]

The area function $\Theta_{P,r}$ is a continuous piecewise polynomial function of degree $n$ if $r$ is not a normal vector of any facet of $P$. 4.15 motivates the following lemma which is the only step where we need the BBaKLP formulas.

Lemma 5.2**.**

Let $r\in\mathds{R}^{n}$ be a vector of unit length such that $\tilde{D}_{v_{i}}(r)$ is non-zero and well-defined, i.e., its numerator and denominator is non-zero. Then

[TABLE]

Proof.

Set $y:=\langle x,r\rangle$. From (13) for $j=0,\dots,n-1$ we have

[TABLE]

and from (14) with $j^{\prime}\geq 0$ we have

[TABLE]

Here $(*)$ and $(+)$ hold since $\mathrm{supp}\,\Theta_{P,r}$ is compact. Thus the claim follows since the set of polynomial functions on a compact set $K$ is dense in $C^{\infty}(K)$. ∎

In the previous proof the BBaKLP formulas were used for all monomials $y^{j}$ ($j\in\mathds{N}_{0}$) and the Weiserstraß Theorem gives the assertion. But the proof of the lemma can be weakened to the Müntz–Szász Theorem [Mün14, Szá16], i.e., only monomials $\{y^{d_{i}}\}_{i\in\mathds{N}}$ with $\sum_{i\in\in\mathds{N}}\frac{1}{d_{i}}=\infty$ (and $d_{1}=0$) are necessary. Additionally, the BBaKLP formulas hold only for polynomials but the previous lemma applies to all $C^{n}$-functions. So we have the following.

Theorem 5.3**.**

Let $\mathcal{A}$ be a (finite-dimensional) vector space of measurable functions on $\mathds{R}$ with basis $\mathsf{A}=\{a_{1},a_{2},\dots\}$ such that $\partial\mathcal{A}\subseteq\mathcal{A}$, i.e., $\partial^{d}\mathcal{A}\subseteq\mathcal{A}$ for all $d\in\mathds{N}$. Let $P\subset\mathds{R}^{n}$ be a polytope with vertices $v_{1},\dots,v_{k}$, $k\geq n+1$, $r\in\mathds{R}^{n}$ be such that it is neither a pole nor a zero of any $\tilde{D}_{v_{i}}(\,\cdot\,)$, and consider the directional moments

[TABLE]

Then $\partial^{n}s$ has an at most $k$-atomic signed representing measure

[TABLE]

supported only at the projections $\langle v_{i},r\rangle$ of the vertices $v_{i}$.

Proof.

Since $s$ has the representing measure $\Lambda_{\Theta_{P,r}}$, the $\partial^{n}s$ has the at most $k$-atomic representing (signed) measure $\partial^{n}\Lambda_{\Theta_{P,r}}=\sum_{i=1}^{k}\tilde{D}_{v_{i}}(r)\cdot\delta_{\langle v_{i},r\rangle}$ by 4.10 and 5.2. ∎

What remains is to extract the positions $\langle v_{i},r\rangle$ from $\partial^{n}s$. If $\mathcal{A}$ consists of polynomials, the generalized eigenvalue problem in 3.1 can be applied. From this we easily get the following corollary, cf. e.g. [GLPR12, Main Theorem]. Note that we propose to replace Prony’s Method/Vandermonde factorization of finite Hankel matrices by the (numerically more stable) generalized eigenvalue problem (as in 3.1), see [GMV99, p. 1225]. For simplicity we assume uniform distribution on $P$. Polynomial distributions on semi-algebraic sets are treated below.

Corollary 5.4**.**

Let $P\subset\mathds{R}^{n}$ be a polytope with vertices $v_{1},\dots,v_{k}$, $k\geq n+1$ and let $r\in\mathds{R}^{n}$ be such that it is neither a pole nor a zero of any $\tilde{D}_{v_{i}}(\,\cdot\,)$, and for $j=0,\dots,2k-n+1$ let $s_{j}=s_{j}(r)$ be the directional moments

[TABLE]

Then the projections $\xi_{i}:=\langle v_{i},r\rangle$ are the eigenvalues of the generalized eigenvalue problem

[TABLE]

Proof.

As in 5.3 $s=(s_{i})_{i=0}^{2k+1}$ has the representing measure $\Lambda_{\Theta_{P,r}}$ and $\partial^{n}s$ has the at most $k$-atomic representing (signed) measure $\partial^{n}\Lambda_{\Theta_{P,r}}=\sum_{i=1}^{k}\tilde{D}_{v_{i}}(r)\cdot\delta_{\langle v_{i},r\rangle}$ by 4.10 and 5.2. By 3.1 the positions $\xi_{i}=\langle v_{i},r\rangle$ are the eigenvalues of the generalized eigenvalue problem (16). ∎

Remark 5.5**.**

Besides the simple proof, the method of derivatives of moments has another advantage. Since 5.2 holds in the distributional sense, 5.3 holds for more general functions $a_{i}$, especially non-polynomial directional moments like in 4.4(b) or (c). However, the generalized eigenvalue problem must then be replaced by a suitable method to determine the atoms $\delta_{\xi_{i}}$ from $\partial^{n}s$.

Remark 5.6**.**

In [GLPR12, Eq. (5)] a “scaled vector of moments” is defined in a similar way as $\partial^{n}s$ in 4.4(a). However, the strength of 4.10, in particular in combination with 4.14, has not been used.

Remark 5.7**.**

With $n+1$ different directions $r$ the vertices can be reconstructed using the previous theorem and $(n+1)(2k-n)+1$ moments are required. If $k$ is unknown, the previous theorem also determines $k$ if sufficiently many directional moments are given.

Now we extend 5.1 to functions $f$:

[TABLE]

i.e., integration by part over $H_{r,x}$.

By linearity of integration and differentiation 5.4 also detects the vertices $v_{i,j}$, $j=1,\dots,d_{i}$, of full-dimensional polytopes $P_{i}\subset\mathds{R}^{n}$, $j=1,\dots,p$, from the moments

[TABLE]

of the simple function

[TABLE]

if the $P_{i}$ or $c_{i}$ are in general position. We say that a set $\{P_{i}\}_{i=1}^{p}$ of polytopes is in general position iff $v_{i,j}\neq v_{i^{\prime},j^{\prime}}$ for all $(i,j)\neq(i^{\prime},j^{\prime})$. Furthermore, we say that $c_{1},\dots,c_{p}$ are in general position iff

[TABLE]

has non-zero mass $\mu(\langle v_{i,j},r\rangle)\neq 0$ for $r\in\mathds{R}^{n}$ in general position, i.e., coefficients in (20) do not cancel out for vertices $v_{i,j}$ with the same projection $\langle v_{i,j},r\rangle$.

Theorem 5.8**.**

Let $P_{i}\subset\mathds{R}^{n}$, $i=1,\dots,p$, be full-dimensional polytopes with vertices $v_{i,j}$, $j=1,\dots,d_{i}$. Let the vertices $v_{i,j}$ or $c_{1},\dots,c_{p}$ be in general position. Let $d:=d_{1}+\dots+d_{p}$. Then for a direction $r\in\mathds{R}^{n}$ in general position the projections $\xi_{i,j}:=\langle r,v_{i,j}\rangle$ are the eigenvalues of the generalized eigenvalue problem

[TABLE]

where $s_{0},\dots,s_{2d-n+1}$ are the directional moments (18) of (19).

Proof.

By linearity of $\partial^{n}$ and 5.2 we have that

[TABLE]

is a (signed) representing measure of $\partial^{n}s$ (4.10). Then $(\partial^{n}\Lambda_{\Theta_{\xi,r}})(\langle r,v_{i,j}\rangle)$ $\neq 0$ for all $i,j$ since the $v_{i,j}$ or $c_{i}$ are in general position. Hence the projections $\langle r,v_{i,j}\rangle$ are the eigenvalues of (21) by 3.1. ∎

Reconstruction of Simple Functions from Moments

We want to adapt 5.8 to simple functions

[TABLE]

of hyperrectangles $\square_{j}$, see 4.16. Similar to polytopes we say that the hyperrectangles $\square_{j}$ are in general position if no two facets of the $\square_{j}$’s lie in a common hyperplane. The $c_{j}$’s are called in general position if $\partial_{i}\Lambda_{\Theta_{\chi,e_{i}}}$ is an at most $2k$-atomic signed measure supported exactly at $p_{j,i,a}$ ($j=1,\dots,k$, $a=0,1$) and $\partial^{\mathds{1}}\Theta_{\chi,r}$ is an at most $k\cdot 2^{n}$-atomic signed measure supported exactly at all $p_{j,i,a}$ ($j=1,\dots,k$, $i=1,\dots,n$, $a=0,1$). We have the following.

Theorem 5.9**.**

Let $k,n\in\mathds{N}$ and

[TABLE]

the simple function of hyperrectangles $\square_{j}$ with $c_{1},\dots,c_{k}$ or $\square_{1},\dots,\square_{k}$ in general position. Consider the moments

[TABLE]

Then for each $i=1,\dots,n$ we have

[TABLE]

i.e., the vertices of the hyperrectangles $\square_{j}$ are contained in the grid

[TABLE]

where the $\xi_{i,j}$ are the $2k$ eigenvalues of the generalized eigenvalue problem

[TABLE]

Proof.

$t=(s_{l\cdot e_{i}})_{l=0}^{4k+1}$ are the moments of the area function $\Theta_{f,e_{i}}$ which has by assumption jumps exactly at the $p_{j,i,l}$’s, $j=1,\dots,k$, $l\in\{0,1\}$. Hence $\partial_{i}t$ is represented by a signed atomic measure supported exactly at the $p_{j,i,l}$’s by 4.10 and the positions are gained from the generalized eigenvalue problem (3.1) ∎

Remark 5.10**.**

For the grid (23) we can then chose an $r\in\mathds{R}^{n}$ in general position such that $\xi\mapsto\langle\xi,r\rangle$ between grid points $\xi$ and their projection $\langle\xi,r\rangle$ is a bijection. Since the $c_{j}$’s or $\square_{j}$’s are in general position we can extract these projections from 5.8 and uniquely recover the vertices of all $\square_{j}$’s. The $c_{j}$’s can then easily (successively) be calculated from evaluation polynomials and $\partial^{\mathds{1}}s$.

Compared to 5.4 and 5.8 we no longer have the disadvantage that we need to chose $n+1$ random directions $r_{i}$. We can choose the directions $e_{1},\dots,e_{n}$ and only $r$ in 5.10 needs to be in general direction but can be chosen based on the grid (23) from the $e_{i}$’s. We need to solve $n$ generalized eigenvalue problems (24) of size at most $(2k+1)\times(2k+1)$. The choice of $e_{i}$ is essential so that we cover $2^{n-1}$ vertices of the same $\square_{j}$ by $\Theta_{\chi,e_{i}}$ at once and hence get $n$ small generalized eigenvalue problems. Only when we cut the vertices of $\square_{j}$ out of the grid (23) we need to go to much higher degrees and have to solve one much larger generalized eigenvalue problem based on 5.8. But better options for cutting the vertices $p_{j,J}$ out of (23) might be possible.

Reconstruction of Measures on Semi-Algebraic Sets

So far we avoided to deal with non-constant densities on bounded sets. Inspired by the work of F. Bréhard, M. Joldes, and J.-B. Lasserre [BJL19] we want to demonstrate how our approach can be applied in this case. This and the previous works [LPHT08], [HK14], [MWHL18] from (optimal) control applications of the moment-SOS-hierarchy were pointed out to us by the authors of [BJL19].

Let $G\subseteq\mathds{R}^{n}$ be a semi-algebraic set and $g\in\mathds{R}[x_{1},\dots,x_{n}]$ such that $\partial G\subseteq\mathcal{Z}(g)$. For $f\in C^{\infty}(\mathds{R}^{n},\mathds{R})$ and $\Lambda\in\mathcal{D}(\mathds{R}^{n})^{\prime}$ we have the Leibniz formula $\partial_{i}(f\cdot\Lambda)=\partial_{i}f\cdot\Lambda+f\cdot\partial_{i}\Lambda$ [Gru09, Lem. 3.7] and if $\Lambda=\chi_{G}$ then $\partial_{i}\chi_{G}$ acts on test functions $\varphi\in\mathcal{D}(\mathds{R}^{n})$ as (weighted) $(n-1)$-dimensional Lebesgue measure supported on $\partial G$, i.e., $(\partial_{i}\chi_{G})(g\cdot\varphi)=0$ for all $\varphi\in\mathcal{D}(\mathds{R}^{n})$ [Gru09, p. 33]. For $g=\sum_{\alpha}g_{\alpha}x^{\alpha}$ we set $g(M):=\sum_{\alpha}g_{\alpha}M_{\alpha}$, where the $M_{\alpha}$ are the shifts from 2.7. Remember the matrix notation $(s,t,\dots,z)_{l}$ from 2.8.

Theorem 5.11** ([BJL19, Thm. 1]).**

Let $G\subseteq\mathds{R}$ be a semi-algebraic set, let $g\in\mathds{R}[x_{1},\dots,x_{n}]$ with $\gamma:=\deg g$ and $\partial G\subseteq\mathcal{Z}(g)$, $p\in\mathds{R}[x_{1},\dots,x_{n}]$ with $d:=\deg p$, and $s_{\alpha}$ the moments of $\exp(p)\cdot\chi_{G}$,

[TABLE]

for all $\alpha\in\mathds{N}_{0}^{n}$ with $|\alpha|\leq k$ for some $k\geq 2d+2\gamma-2$. The following are equivalent:

1. i)

$p=\sum_{\alpha\in\mathds{N}_{0}:|\alpha|\leq d}c_{\alpha}\cdot x^{\alpha}$. 2. ii)

For each $i=1,\dots,n$ let $\alpha^{(1)},\alpha^{(2)},\dots,\alpha^{(m)}$ with $m=\binom{n+d-1}{n}$ denote an enumeration of $\alpha=(\alpha_{1},\dots,\alpha_{n})\in\mathds{N}_{0}^{n}$ with $|\alpha|\leq d$ and $\alpha_{i}\geq 1$. The kernel of

[TABLE]

is spanned by $(1,-\alpha_{i}^{(1)}\cdot c_{\alpha^{(1)}},\dots,-\alpha_{i}^{(m)}\cdot c_{\alpha^{(m)}})$ for every $i=1,\dots,n$.

$c_{0}$* is determined by normalization. If $g\geq 0$ on $G$ then $k\geq 2d+\gamma-2$ is sufficient.*

Proof.

Note that $s$ is represented by $\exp(p)\cdot\chi_{G}$, $\partial_{i}s$ is presented by $\partial_{i}(\exp(p)\cdot\chi_{G})=\partial_{i}p\cdot\exp(p)\cdot\chi_{G}+\exp(p)\cdot\partial_{i}\chi_{G}$ and since $(\partial_{i}\chi_{G})(g\cdot\varphi)=0$ for all $\varphi\in\mathcal{D}(\mathds{R}^{n})$ we finally have that $g(M)\partial_{i}s$ is represented by $g\cdot\partial_{i}p\cdot\exp(p)\cdot\chi_{G}$.

(ii) $\Rightarrow$ (i): So by the previous note $\partial_{i}p$, i.e., $(1,-\alpha_{i}^{(1)}\cdot c_{\alpha^{(1)}},\dots,-\alpha_{i}^{(m)}\cdot c_{\alpha^{(m)}})$, is in the kernel and all $c_{\alpha}$’s with $\alpha\neq 0$ are determined.

(i) $\Rightarrow$ (ii): Again $\partial_{i}p$, i.e., $(1,-\alpha_{i}^{(1)}\cdot c_{\alpha^{(1)}},\dots,-\alpha_{i}^{(m)}\cdot c_{\alpha^{(m)}})$, is in the kernel of (25). It is sufficient to show that $(g(M)M_{\alpha^{(1)}-e_{i}}s,\ \dots,\ g(M)M_{\alpha^{(m)}-e_{i}}s)_{k-d}$ is full dimensional to show that the kernel is one-dimensional. Assume the columns of $(g(M)M_{\alpha^{(1)}-e_{i}}s,\ \dots,\ g(M)M_{\alpha^{(m)}-e_{i}}s)_{k-d}$ are linearly dependent, then by the linearity of the shift $M_{\alpha}$ also the columns of

[TABLE]

are linearly dependent. But ($*$) is the Hankel matrix of $g^{2}(M)s$, a moment sequence with representing measure $g^{2}\cdot\exp(p)\cdot\chi_{G}$, i.e., has full rank. This proves that the kernel of (25) is one-dimensional.

If $g\geq 0$ on $G$, squaring $g$ in “(ii) $\Rightarrow$ (i)” is not necessary and linear independence already holds for $k\geq 2d+\gamma-1$. ∎

The bound $k\geq 2d+2\gamma-2$, resp. $k\geq 2d+\gamma-2$, comes from the maximal $\alpha$, i.e., $s_{\alpha}$, needed to construct (25). If $d=\deg p$ is unknown, then the previous theorem also recovers $d$ if $k$ is large enough. For $k\geq 2d+2\gamma-2$ the kernel of (25) is one-dimensional, i.e., determines $d$ as $\max_{c_{\alpha}\neq 0}|\alpha|$. For $k<2d+2\gamma-2$ (resp. $2d+\gamma-2$) (25) is full rank.

In [BJL19] also the problem of finding $g$ from $s=(s_{\alpha})$ for an unknown $G$ is addressed, but then all moments $s_{\alpha}$ are necessary.

6. Gaussian Mixtures

One component

For a Gaussian distribution $g(x)=c\cdot\exp(-\frac{a}{2}(x-b)^{2})$ on $\mathds{R}$ we have

[TABLE]

So integration over $x^{i}\cdot g^{\prime}(x)$ gives

[TABLE]

see also [AFS16, Eq. (5)]. This implies the following result.

Lemma 6.1** ([AFS16, Prop. 1]).**

Let $k\in\mathds{N}$, $k\geq 2$, be a natural number and $s=(s_{0},s_{1},\dots,s_{k})$ be a real sequence with $s_{0}\neq 0$. The following are equivalent:

1. i)

$s$* is the moment sequence of the Gaussian distribution $c\cdot\exp(-\frac{a}{2}(x-b)^{2})$ with $a,b,c\in\mathds{R}$, $a>0$, $c\neq 0$, i.e., $s_{i}=\int x^{i}\cdot c\cdot\exp(-\frac{a}{2}(x-b)^{2})\leavevmode\nobreak\ \mathrm{d}x$.* 2. ii)

There are $a,b\in\mathds{R}$ with $a>0$ such that the matrix

[TABLE]

has rank two with kernel $(1,-ab,a)^{T}\cdot\mathds{R}$.

In this case, one has $a=\frac{s_{0}^{2}}{s_{0}s_{2}-s_{1}^{2}}$, $b=\frac{s_{1}}{s_{0}}$ and $c=s_{0}\cdot\sqrt{\frac{a}{\pi}}$.

Proof.

While (i) $\Rightarrow$ (ii) is clear, we show (ii) $\Rightarrow$ (i) by induction on $i$. Since $0\neq s_{0}=c\cdot\int e^{-a(x-b)^{2}}\leavevmode\nobreak\ \mathrm{d}x$ for $c=s_{0}\cdot\sqrt{\frac{a}{\pi}}$ and $s_{-1}:=0$, we have by (ii), (25), (26) and the induction hypothesis that

[TABLE]

i.e., $s_{i+1}$ is the $(i+1)$-th moment of $c\cdot\exp(-a(x-b)^{2})$. ∎

On $\mathds{R}^{n}$ we have the following.

Theorem 6.2**.**

Let $n\in\mathds{N}$, $A=(a_{1},\dots,a_{n})=(a_{i,j})_{i,j=1}^{n}\in\mathds{R}^{n\times n}$ be a symmetric and positive definite matrix, $b\in\mathds{R}^{n}$, $c\in\mathds{R}$, $c\neq 0$, and $k\in\mathds{N}$ with $k\geq 2$. Set

[TABLE]

For a multi-indexed real sequence $s=(s_{\alpha})_{\alpha\in\mathds{N}_{0}^{n}:|\alpha|\leq k}$ the following are equivalent:

1. i)

$s$* is the moment sequence of $\Lambda_{g}$, i.e., $s_{\alpha}=\int x^{\alpha}\cdot g(x)\leavevmode\nobreak\ \mathrm{d}\lambda^{n}(x)$ for all $\alpha\in\mathds{N}_{0}^{n}$ with $|\alpha|\leq k$.* 2. ii)

For $i=1,\dots,n$ the matrix $(\partial_{i}s,s,M_{e_{1}}s,\dots M_{e_{n}}s)_{k-1}$ has the $1$-dimensional kernel

[TABLE]

Proof.

For $i=1,\dots,n$ we have

[TABLE]

(i) $\Rightarrow$ (ii): From ($*$) we find that (27) is contained in the kernel of the matrix $(\partial_{i}s,s,M_{e_{1}}s,\dots,M_{e_{n}}s)_{k-1}$. It suffices to show that the kernel of the matrix $(\partial_{i}s,s,M_{e_{1}}s,\dots,M_{e_{n}}s)_{1}$ is at most one-dimensional. Consider

[TABLE]

the Hankel matrix of $L_{s}|_{\mathds{R}[x_{1},\dots,x_{n}]_{\leq 2}}$. Let $d=(d_{0},\dots,d_{n})\in\ker H$. Then $0=L_{s}(\langle d,(1,x_{1},\dots,x_{n})\rangle^{2})=\int(d_{0}+d_{1}x_{1}+\dots+d_{n}x_{n})^{2}\leavevmode\nobreak\ \mathrm{d}\Lambda_{g}(x)$ implies $d=0$, i.e., $H$ has full rank $n+1$. Therefore $(\partial_{i}s,s,M_{e_{1}}s,\dots,M_{e_{n}}s)_{1}$ has rank at least $n+1$ since it has $H$ as submatrix. Its kernel can thus be at most one-dimensional.

(ii) $\Rightarrow$ (i): Let $O\in\mathds{R}^{n\times n}$ be an orthogonal matrix such that $O\cdot A\cdot O^{T}=\mathrm{diag}\,(\lambda_{1},\dots,\lambda_{n})$, $\lambda_{i}>0$. The coordinate change on $\mathds{R}^{n}$ given by $y=Ox$ induces a linear transformation on the space of moment sequences. Let $t=(t_{\alpha})_{|\alpha|\leq k}$ be the moment sequence obtained from $s$ via this transformation. A straight-forward calculation shows that

[TABLE]

where $\tilde{b}=Ob$. This means that we are in the 1-dimensional setting

[TABLE]

where the $1$-dimensional assertion holds by 6.1. Hence, $t=(t_{\beta})$ is represented by $t_{0}\cdot\frac{\sqrt{\lambda_{1}\cdots\lambda_{n}}}{(\pi)^{n/2}}\prod_{i=1}^{n}e^{-\frac{\lambda_{i}}{2}(y_{i}-\tilde{b}_{i})^{2}}$. The inverse transformation $x=O^{T}y$ together with $\lambda_{1}\cdots\lambda_{n}=\det(A)$ gives the $n$-dimensional assertion. ∎

Hence, the previous theorem provides an easy way to determine $A\in\mathds{R}^{n\times n}$ and $b\in\mathds{R}^{n}$ from the moments $s_{\alpha}$.

Algorithm 6.3**.**

1. Input:

$k\in\mathds{N}$, $k\geq{2}$; $s=(s_{\alpha})_{\alpha\in\mathds{N}_{0}^{n}:|\alpha|\leq k}$. 2. Step 1:

For $i=1,\dots,n$:

1. a)

Calculate $\beta_{i}$ and $a_{i}=(a_{i,1},\dots,a_{i,n})$ from

[TABLE]

If the kernel is not one-dimensional, then $s$ is not represented by one Gaussian distribution. 2. b)

Check: $(1,-\beta_{i},a_{i,1},\dots,a_{i,n})\in\ker(\partial_{i}s,s,M_{e_{1}}s,\dots M_{e_{n}}s)_{k-1}$?

If FALSE: $s$ is not represented by one Gaussian distribution. 3. Step 2:

Check: $A=(a_{i,j})_{i,j=1}^{n}$ is symmetric and positive definite?

If FALSE: $s$ is not represented by one Gaussian distribution. 4. Step 3:

Calculate $b=A^{-1}\cdot(\beta_{1},\dots,\beta_{n})^{T}$ and $c=\frac{\sqrt{\det(A)}}{\pi^{n/2}}\cdot s_{0}$. 5. Out:

“$s$* is represented by a Gaussian distribution”: TRUE or FALSE. If TRUE: $A$, $b$, $c$.*

With

[TABLE]

we get a result similar to 5.11 but with integration over $\mathds{R}^{n}$ instead of a semi-algebraic set $G$.

Theorem 6.4**.**

Let $k,d\in\mathds{N}$ with $k\geq 4d-2$ and $s=(s_{0},\dots,s_{k})$ be a real sequence with $s_{0}\neq 0$. The following are equivalent:

1. i)

$s$* is the moment sequence of the distribution $c\cdot\exp\left(\frac{-a}{2d}(x-b)^{2d}\right)$ with $a,b,c\in\mathds{R}$, $a>0$, $c\neq 0$.* 2. ii)

There are $a,b\in\mathds{R}$ with $a>0$ such that the matrix

[TABLE]

has a one-dimensional kernel spanned by

[TABLE]

and $s_{i}$ is the $i$-th moment of $c\cdot\exp\left(\frac{-a}{2d}(x-b)^{2d}\right)$ for $i=0,\dots,2d-2$.

In this case $c=\sqrt[2d]{\frac{a}{2d}}\cdot\frac{s_{0}}{2\cdot\Gamma(1+\frac{1}{2d})}$.

Proof.

Similar to the proof of 6.1 using (27) instead of (26) in the induction. The formula for $c$ follows from $\int_{\mathds{R}}\exp(-x^{2d})\leavevmode\nobreak\ \mathrm{d}x=2\cdot\Gamma(1+\frac{1}{2d})$, $d\in\mathds{N}$. ∎

Multiple components in dimension one with same variance.

While we fully characterized all moment sequences represented by one Gaussian distribution and showed how to determine the parameters, let us investigate mixtures with more than one component. In this study the elementary symmetric polynomials play a crucial role.

Definition 6.5**.**

For $k,j\in\mathds{N}$ with $j\leq k$ we denote by

[TABLE]

the elementary symmetric polynomials.

The elementary symmetric polynomials have the following property.

Lemma 6.6** (Vieta’s Formulas).**

Let $k\in\mathds{N}$ and $b_{1},\dots,b_{k}\in\mathds{R}$ be pairwise different points. For $v_{1},\dots,v_{k}\in\mathds{R}$ the following are equivalent:

1. i)

[TABLE] 2. ii)

$v_{l}=(-1)^{l}\cdot\sigma_{l}(b_{1},\dots,b_{k})$* for all $l=1,\dots,k$.* 3. iii)

$\mathcal{Z}(p)=\{b_{1},\dots,b_{k}\}$* with*

[TABLE]

Proof.

Follows directly from

[TABLE]

Since we assume all Gaussian distributions to have the same variance, we introduce the following convenient operator.

Definition 6.7**.**

Let $L:\mathds{R}[x]_{\leq d}\to\mathds{R}$ be a linear functional ($s\in\mathds{R}^{d+1}$) with $d\in\mathds{N}\cup\{\infty\}$ and a differentiable function $f\in C^{1}(\mathds{R},\mathds{R})$. For $a\in\mathds{R}\setminus\{0\}$ we define

[TABLE]

Note, that we use $\Delta_{a}$ as an operator acting on functionals and on functions to emphasize the close connection between the operations performed on $L$ and measures $\mu=\Lambda_{f}$ provided by 4.10.

$\Delta_{a}$ has the following properties (Lemmas 6.8–6.11).

Lemma 6.8**.**

Let $L:\mathds{R}[x]_{\leq d}\to\mathds{R}$ be a linear functional with $d\in\mathds{N}\cup\{\infty\}$ and $a\neq 0$. Then

[TABLE]

Proof.

$n=1$: $0=(\Delta_{a}L)(1)=\frac{1}{a}((\partial+aM_{1})L)(1)=\frac{1}{a}(-L(\partial 1)+a\cdot L(x))=L(x)$.

$n\to n+1$: With $L(x^{i})=0$ for all $i=1,\dots,n$ it follows that $0=(\Delta_{a}L)(x^{n})=\frac{1}{a}((\partial+aM_{1})L)(x^{n})$ $=\frac{1}{a}(-n\cdot L(x^{n-1})+a\cdot L(x^{n+1}))=L(x^{n+1})$. ∎

Lemma 6.9**.**

Let $k\in\mathds{N}$, $a\in\mathds{R}\setminus\{0\}$, and

[TABLE]

for some $b_{1},\dots,b_{k}\in\mathds{R}$ pairwise different and $c_{1},\dots,c_{k}\in\mathds{R}\setminus\{0\}$. Then

[TABLE]

for every $l\in\mathds{N}_{0}$.

Proof.

Follows by induction on $l$. $l=0$ is clear. We have to show $l\to l+1$:

[TABLE]

Lemma 6.10**.**

Let $d\in\mathds{N}$, $a>0$, $b\in\mathds{R}$, and $\mathsf{A}=\{1,x,\dots,x^{d}\}$. Define

[TABLE]

i.e., $t_{a}(b)$ is the moment vector of the moments $s_{0},\dots,s_{d}$ of $\exp\left(-\frac{a}{2}(x-b)^{2}\right)$. Then there is an invertable matrix $\mathfrak{M}=\mathfrak{M}(a)\in\mathds{R}^{(d+1)\times(d+1)}$ such that

[TABLE]

and it follows that

[TABLE]

Proof.

Since

[TABLE]

we find that the $i$-th entry in $t_{a}(b)$ is a polynomial of degree $i$ in $b$. The coordinate change to $\mathsf{A}=\{1,b,\dots,b^{i}\}$ is $\mathfrak{M}$. The second statement follows immediately from

[TABLE]

i.e., $\Delta_{a}^{l}t_{a}(b)=b^{l}t_{a}(b)$ and $\mathfrak{M}\Delta_{a}^{l}t_{a}(b)=b^{l}\mathfrak{M}t_{a}(b)=b^{l}s_{\mathsf{A}}(b)=M_{l}s_{\mathsf{A}}(b)$. ∎

Lemma 6.11**.**

Let $k\in\mathds{N}$ and $F(x)$ be the Gaussian mixture

[TABLE]

for $b_{1},\dots,b_{k}\in\mathds{R}$ pairwise different and $c_{1},\dots,c_{k}\in\mathds{R}\setminus\{0\}$. Let

[TABLE]

be the moments of $F(x)$ up to degree $2k-2$. The following matrix has full rank:

[TABLE]

Proof.

Take $\mathfrak{M}\in\mathds{R}^{k\times k}$ from 6.10 and set $\tilde{s}:=\mathfrak{M}(s_{0},\dots,s_{k-1})$. Then

[TABLE]

is full rank as in the one-dimensional case. ∎

With these properties of $\Delta_{a}$ we can characterize moments sequences which are represented by (30) and determine the parameters $b_{i}$ if $a>0$ is known.

Theorem 6.12**.**

Let $k,d\in\mathds{N}$ with $d\geq k$, $s=(s_{0},s_{1},\dots,s_{d})\in\mathds{R}^{d+1}$. The following are equivalent:

1. i)

For $a>0$, $c_{1},\dots,c_{k}\in\mathds{R}\setminus\{0\}$, and $b_{1},\dots,b_{k}\in\mathds{R}$ pairwise different we have that $s=(s_{0},\dots,s_{d})$ has the representing measure $\Lambda_{F}$ with

[TABLE] 2. ii)

For $a>0$ and $b_{1},\dots,b_{k}\in\mathds{R}$ pairwise different we have that

[TABLE]

If additionally $d\geq 2k$, then both are equivalent to the following:

1. iii)

For $a>0$ and $b_{1},\dots,b_{k}\in\mathds{R}$ pairwise different we have that

[TABLE] 2. iv)

For $a>0$ and $b_{1},\dots,b_{k}\in\mathds{R}$ pairwise different we have that

[TABLE]

and $\mathcal{Z}(p)=\{b_{1},\dots,b_{k}\}$ for

[TABLE]

If one of the equivalent statements (i)–(iv) and $\mathcal{H}(\mathfrak{M}(a)s)\succeq 0$ hold, then $c_{i}>0$.

Proof.

Using $\mathfrak{M}(a)$ from 6.10 transforms each statement (i)–(iv) into the corresponding one-dimensional statements for Dirac measures (i’)–(iv’). Then the equivalence of all statements (i)–(iv) follows from the equivalence of (i’)–(iv’). ∎

Remark 6.13**.**

From the proof it is evident that by a coordinate change induced by $\mathfrak{M}(a)$ from 6.10 the one-dimensional case of Gaussian mixtures with the same known variance is the same as the one-dimensional case of Dirac measures. This can also be seen from $\Delta_{a}^{l}\xrightarrow{a\to\infty}M_{l}$.

So the highly non-linear problem of finding $k$ and $b_{1},\dots,b_{k}$ from the moments $s$ reduces to the linear problem of calculating the kernel of (31) and the well-studied problem of finding all roots of a univariate polynomial (32). The coefficients $c_{1},\dots,c_{k}$ can then be determined by linear algebra.

But 6.12 only applies if we know $a$ beforehand. We therefore have to determine $a>0$ from $s=(s_{0},\dots,s_{d})$ as well. Set $\tilde{\Delta}_{a}:=(\partial+aM_{1})$, i.e., $\tilde{\Delta}_{a}=a\cdot\Delta_{a}$, and observe

[TABLE]

holds for some $\alpha_{i,j}\in\mathds{R}$ and all $f\in C^{l}(\mathds{R},\mathds{R})$. Applying this to (30), i.e., $f=F$, shows that the linear dependence

[TABLE]

from 6.6, resp. 6.12, implies the linear dependence of $\{x^{i}\partial^{j}F\,|\,0\leq i,j$ and $i+j\leq k\}$,

[TABLE]

for some $\beta_{i,j}\in\mathds{R}$, and therefore also the moments $\{M_{i}\partial^{j}s\,|\,0\leq i,j$ and $i+j\leq k\}$,

[TABLE]

Let us have a look at a small example.

Example 6.14**.**

For $k=2$ in (30) we have

[TABLE]

i.e., the matrix

[TABLE]

contains the following vector in its kernel:

[TABLE]

For sufficiently large $d\in\mathds{N}$ the kernel is one-dimensional. Hence,

[TABLE]

and by Vieta’s formulas (6.6) we have that $b_{1}$ and $b_{2}$ are the zeros of

[TABLE]

The previous example provides one way to find $a>0$. It determines $a$ uniquely (and the $b_{1},\dots,b_{k}$ simultaneously) but with the cost that more moments are required than in 6.12. In 6.12 we need $2k$ moments, while for the generalized method of the previous example the matrix must be of size $\frac{k(k+1)}{2}\times K$ with $K\geq\frac{k(k+1)}{2}-1$. Hence, moments of degree at least $\frac{k^{2}+3k-2}{2}$ are required since the last line contains $M_{k}s_{K}=s_{K+k}$.

However, with the following approach we also get $a$ from 6.12.

Definition 6.15**.**

Let $k\in\mathds{N}$ and $s=(s_{0},\dots,s_{2k})\in\mathds{R}^{2k+1}$. We define

[TABLE]

Example 6.16**.**

1. a)

For $s=(s_{0},s_{1},s_{2})\in\mathds{R}^{3}$, i.e., $k=1$, we have

[TABLE] 2. b)

For $s=(s_{0},s_{1},s_{2},s_{3},s_{4})\in\mathds{R}^{3}$, i.e., $k=2$, we have

[TABLE]

Lemma 6.17**.**

Let $k\in\mathds{N}$ and $s=(s_{0},\dots,s_{2k})\in\mathds{R}^{2k+1}$. The following holds:

1. i)

$\mathfrak{p}_{s}(x)\in\mathds{R}[x]_{\leq\frac{k(k+1)}{2}}$. 2. ii)

If $s$ is represented by $\Lambda_{F}$ with

[TABLE]

for some $a>0$, $c_{1},\dots,c_{k}\in\mathds{R}\setminus\{0\}$, and $b_{1},\dots,b_{k}\in\mathds{R}$ pairwise different. Then

[TABLE]

Proof.

This follows immediately from 6.15 and 6.12. ∎

The previous lemma combined with 6.12 provides the following algorithm to determine a Gaussian mixture representation of $s$ with equal variance for each Gaussian component.

Algorithm 6.18**.**

1. Input:

$k\in\mathds{N}$* and $s=(s_{0},s_{1},\dots,s_{d})\in\mathds{R}^{d+1}$ with $d\geq 2k$.* 2. Step 1:

1. a)

Calculate $\mathfrak{p}_{s}(a):=a^{\frac{k(k+1)}{2}}\cdot\det((s,\Delta_{a}s,\dots,\Delta_{a}^{k}s)_{k})$. 2. b)

Calculate $Z:=\mathcal{Z}(\mathfrak{p}_{s})\cap\mathds{R}_{>0}=\{a_{1},\dots,a_{l}\}$.

If $Z$ is empty, $s$ has no $k$-Gaussian mixtures with equal variance. 3. Step 2:

For $i=1,\dots,l$:

1. a)

Calculate $v_{1},\dots,v_{k}\in\mathds{R}$ from (31):

[TABLE]

If ($*$) does not hold: $a_{i}$ is not a variance for $s$. Goto $i+1$. 2. b)

Calculate zeros $\mathcal{Z}(p)=\{b_{1},\dots,b_{k}\}$ of (32):

[TABLE]

If $p$ has complex solutions: $a_{i}$ is not a variance for $s$. Goto $i+1$. 4. Step 3:

Calculate $c_{1},\dots,c_{k}\in\mathds{R}$ from the $t_{a_{i}}(b_{j})$’s in (29):

[TABLE] 5. Out:

$a>0$, $b_{1},\dots,b_{k}\in\mathds{R}$, and $c_{1},\dots,c_{k}\in\mathds{R}$.

This algorithm can of course be modified to determine $k$ as well. Add an outer loop testing 6.18 for $k=1,\dots,\lceil\frac{d}{2}\rceil$.

Multiple components in dimension one.

Now we want to investigate the one-dimensional case with $a_{1},\dots,a_{k}>0$ arbitrary (e.g., pairwise different). For $k=2$ we have the problem already considered by Pearson [Pea94].

Example 6.19**.**

Let $k=2$, $a_{1},a_{2}>0$ with $a_{1}\neq a_{2}$ and $b_{1},b_{2}\in\mathds{R}$. For

[TABLE]

we have that $\{F(x),xF(x),\partial F(x),x^{2}F(x),x\partial F(x),\partial^{2}F(x)\}$ are linearly independent. But adding $\{x^{3}F(x),x^{2}\partial F(x),x\partial^{2}F(x)\}$ (without $\partial^{3}F(x)$) makes the system linearly dependent:

[TABLE]

We have a one-dimensional solution set spanned by

[TABLE]

So $a_{1}$ and $a_{2}$ are the zeros of

[TABLE]

by the Vieta’s Formulas (6.6).

One might to be seduced by this example and the opinion that by replacing the restriction $a_{1}=\dots=a_{k}=a$ by arbitrary $a_{i}>0$ that less Gaussian distribution are required. But 6.22 shows that there are moment sequences with very large mixture Carathéodory numbers.

Multi-dimensional Gaussian mixtures.

So far we only dealt with the one-dimensional case of Gaussian mixture reconstruction from moments. And this was even done with the restriction $a_{1}=\dots=a_{k}=a>0$. In [dDK19] we proved new lower bounds for the Carathéodory numbers for Dirac measures which grow asymptotically close to the Richter upper bound. Now we show that for Gaussian mixtures the same lower bounds hold even when arbitrary variances are allowed.

Before we can state our last main theorem, we need the following definition.

Definition 6.20**.**

Let $\mathcal{A}$ be a finite-dimensional vector space of measurable functions on a measurable space $(\mathcal{X},\mathfrak{A})$ and $\delta_{\sigma,\xi}$ probability measures as in 2.9. A function $a\in\mathcal{A}$ is called non-negative of highest order (with respect to the measures $\delta_{\sigma,\xi}$) if $a\geq 0$ and for any sequence $(c_{i},\sigma_{i},\xi_{i})_{i\in\mathds{N}}\subseteq\mathds{R}_{\geq 0}\Sigma\times\mathcal{X}$ with

[TABLE]

there exists a subsequence $(i_{j})_{j\in\mathds{N}}$ with one of the following properties:

1. i)

$\sigma_{i_{j}}\xrightarrow{j\to\infty}\sigma_{0}$ and $\xi_{i_{j}}\xrightarrow{j\to\infty}\xi\in\mathcal{Z}(a)$, or 2. ii)

$\int_{\mathcal{X}}b(x)\leavevmode\nobreak\ \mathrm{d}(c_{i_{j}}\cdot\delta_{i_{j}})(x)\xrightarrow{j\to\infty}0$ for all $b\in\mathcal{A}$.

Note, being of highest order depends in general on the measures $\delta_{\sigma,\xi}$. The following are examples for non-negative polynomials of highest order.

Example 6.21**.**

Let $d,n\in\mathds{N}$, $\mathcal{X}=\mathds{R}^{n}$, and $\mathcal{A}=\mathds{R}[x_{1},\dots,x_{n}]_{\leq 2d}$. Let $p\in\mathcal{A}$ be non-negative, with finitely many zeros and without zeros at infinity (its homogenization has no zeros with $x_{0}=0$). Then $p$ is non-negative of highest order with respect to Gaussian or log-normal measures. In particular

[TABLE]

is non-negative of highest order.

Recall from [dDS18b] that with $\mathcal{X}\subseteq\mathds{R}^{n}$ open and $\mathcal{A}$ a finite-dimensional space of differentiable functions on $\mathcal{X}$, then $\mathcal{N}_{\mathsf{A}}\in\mathds{N}$ is the smallest $k\in\mathds{N}$ such that $DS_{k,\mathsf{A}}(C,X)$ has full rank for some $(C,X)\in\mathds{R}_{\geq 0}^{k}\times\mathcal{X}^{k}$ where

[TABLE]

with $C=(c_{1},\dots,c_{k})$ and $X=(x_{1},\dots,x_{k})$.

Theorem 6.22**.**

Let $(\mathcal{X},\mathfrak{A})$ be a measurable space, $\mathcal{A}$ be a finite-dimensional space of measurable functions on $(\mathcal{X},\mathfrak{A})$ with an $e\in\mathcal{A}$ such that $e\geq 1$ on $\mathcal{X}$, and $\delta_{\sigma,\xi}$ probability measures on $\mathcal{X}$ as in 2.9. Let $a\in\mathcal{A}$ be non-negative of highest order with finitely many zeros $\mathcal{Z}(a)=\{x_{1},\dots,x_{k}\}$. Then there exists a moment sequence $s\in\mathrm{int}\,\mathcal{S}_{\mathsf{A}}=\mathrm{int}\,\mathcal{T}_{\mathsf{A}}$ with

[TABLE]

If additionally $\mathcal{X}\subseteq\mathds{R}^{n}$ is open, $n\in\mathds{N}$, and $\mathcal{A}$ is $r$-differentiable with $r>\dim\mathcal{A}-\mathcal{N}_{\mathsf{A}}\cdot(n+1)$, then $s$ has an open neighborhood $U$ such that (33) holds for all $s^{\prime}\in U$.

Proof.

Let $K:=\dim\mathrm{lin}\,\{s_{\mathsf{A}}(x_{i})\,|\,i=1\dots,k\}\leq k$ and $s^{(0)}:=\sum_{i=1}^{k}s_{\mathsf{A}}(x_{i})$. Then $L_{s^{(0)}}(a)=0$ and by [dDS18b, Thm. 18] (2.4) we have $\mathcal{C}_{\mathsf{A}}(s^{(0)})=K$.

Let $(s^{(i)})_{i\in\mathds{N}}\subset\mathrm{int}\,\mathcal{S}_{\mathsf{A}}$ be such that $s^{(i)}\to s^{(0)}$ as $i\to\infty$. By [dD19, Thm. 17(ii)] (2.11) any $s^{(i)}$, $i\geq 1$, has a mixture representation

[TABLE]

with $K_{i}=\mathcal{C}_{\mathsf{A}}^{M}(s^{(i)})\leq\dim\mathcal{A}$ (i.e., $K_{i}$ are minimal), $c_{i,j}>0$, $\sigma_{i,j}\in\Sigma$, and $x_{i,j}\in\mathcal{X}$. Since $K_{i}\in\mathds{N}$ we have $K_{0}:=\liminf_{i\to\infty}K_{i}$ and after choosing a subsequence of $(s^{(i)})_{i\in\mathds{N}}$ we can assume that $K_{i}=K_{0}$ for all $i\in\mathds{N}$.

Let us show that $K_{0}\geq K$ holds. Since $a$ is non-negative of highest order, we can assume that the $(c_{i,j},\sigma_{i,j},\xi_{i,j})$ fulfill (i) or (ii) in 6.20 by taking a subsequence $(i_{l})_{l\in\mathds{N}}$. By reordering the $j$’s in ($*$) we can assume that (i) holds for all $j=1,\dots,M$ and (ii) for all $j=M+1,\dots,K_{0}$. Since $c_{i,j}\geq 0$ and $c_{i,1}+\dots+c_{i,K_{0}}=L_{s^{(i)}}(e)=k$ we can assume that $c_{i,j}\xrightarrow{i\to\infty}c_{j}$ for all $j=1,\dots,M$. But (ii) implies

[TABLE]

and therefore we have

[TABLE]

i.e., $K\leq M\leq K_{0}$. Hence $K_{0}\geq K$ implies that all $s^{(i)}$ fulfill $\mathcal{C}_{\mathsf{A}}^{M}(s^{(i)})=K_{0}\geq K$.

If $\mathcal{A}$ are $r$-differentiable functions, then the sequence $(s^{(i)})_{i\in\mathds{N}}$ can be chosen to contain only regular moment sequences by Sard’s Theorem [Sar42] (see [dDS18b]). Hence, for each $i\geq N$ there is an open neighborhood $U_{i}$ of $s^{(i)}$ such that all $s^{\prime}\in U_{i}$ fulfill $\mathcal{C}_{\mathsf{A}}^{M}(s^{(i)})=\mathcal{C}_{\mathsf{A}}^{M}(s^{\prime})$. ∎

So from the proof it is evident that the constructed $s$ with (33) is close to the boundary of the moment cone, more precisely close to the boundary face represented by $a\in\mathcal{A}$. [dD19], [dDK19], 6.22, and 6.21 explicitly provide the following.

Corollary 6.23**.**

Let $d\in\mathds{N}$ and $\mathcal{X}=\mathds{R}$. For the one-dimensional Gaussian (and log-normal) measures with $\mathcal{A}=\mathds{R}[x]_{\leq d}$ we have

[TABLE]

Proof.

6.21 and 6.22 gives the lower bound and [dD19, Cor. 36] the upper bound. ∎

6.23 (and 6.26) explains why we only discussed the reconstruction of one-dimensional Gaussian mixtures with equal variances $a_{1}=\dots=a_{k}=a>0$ at the beginning of this section. There are moment sequences where it is sufficient to represent them by mixtures of $\delta_{\sigma_{i},\xi_{i}}$ with $\sigma_{1}=\dots=\sigma_{k}$ and relaxation of this restriction does not improve the required number of components. Especially in higher dimensions we will see that even in the case of $\mathcal{A}=\mathds{R}[x_{1},\dots,x_{n}]_{\leq 2d}$ with Gaussian measures the number of components becomes very large, close to $\dim\mathcal{A}=\binom{n+2d}{n}$, see 6.27.

Example 6.24** ($\mathcal{X}=\mathds{R}^{2}$).**

Let $\mathcal{X}=\mathds{R}^{2}$ and $\mathcal{A}=\mathds{R}[x_{1},x_{2}]_{\leq d}$, $d\in 2\mathds{N}$. By a rotation of $\mathds{P}^{n}$ we can assume that for homogeneous polynomials in $\mathds{R}[x_{0},x_{1},x_{2}]_{=d}$ with finitely many zeros no zero is at infinity ($x_{0}=0$).

1. a)

$d=4$: The Motzkin polynomial [Mot67] has $6$ projective zeros, is non-negative of highest order, and the point evaluations at these zeros are linearly independent, see [dDS18b, Exm. 31]. So $\mathcal{C}^{M}_{\mathsf{A}_{2,4}}\geq 6$, i.e., there is a moment sequence/functional on $\mathds{R}[x_{1},x_{2}]_{\leq 4}$ which can be represented by a sum of $6$ Gaussians but not less. The upper bound for the Dirac measures in the projective case is also $6$ [Rez92]. 2. b)

$d=6$: The Robinson polynomial [Rob69] has $10$ projective zeros, is non-negative of highest order and all point evaluations at these zeros are also linearly independent, see [dDS18b, p. 1635]. So $\mathcal{C}^{M}_{\mathsf{A}_{2,6}}\geq 10$. Note, that for Dirac measures we have the Carathéodory number $11$ in the projective case, see [Kun14]. 3. c)

$d=10$: The Harris polynomial [Har99] has $30$ zeros, is non-negative of highest order, and the point evaluations at these zeros are linearly independent, see [dDS18b, Exm. 63]. Hence, $\mathcal{C}^{M}_{\mathsf{A}_{2,10}}\geq 30$. An upper bound for Dirac measures in the projective setting is $32$, see [dDS18b, Exm. 63]. 4. d)

$d\in 2\mathds{N}$: In [RS18, Lem. 8.6] it was shown that the point evaluations on the grid

[TABLE]

are linearly independent on $\mathcal{A}=\mathds{R}[x_{1},x_{2}]_{\leq 2d}$. Hence $\mathcal{C}^{M}_{\mathsf{A}_{2,2d}}\geq d^{2}$. Additionally, it was shown that $\mathcal{C}_{\mathsf{A}_{2,2d+1}}\leq\frac{3}{2}d(d+1)+1$ holds. With $\mathcal{C}_{\mathsf{A}_{2,2d}}\leq\mathcal{C}_{\mathsf{A}_{2,2d+1}}$, 6.22, and [dD19, Thm. 35] we have

[TABLE]

In [dDK19] the point evaluation on the grid was extended to higher dimensions and improved lower bounds where found. In fact, the following result was shown.

Proposition 6.25** ([dDK19, Prop. 5.3]).**

Let $n,d\in\mathds{N}$, $k\in\{0,1\}$, $\mathcal{X}=\mathds{R}^{n}$, and $G=\{1,\dots,d\}^{n}$. Then

[TABLE]

supported on the grid $G$ with the representing measure $\mu=\sum_{x\in G}\delta_{x}$ has the Carathéodory number

[TABLE]

Since the grid $G=\{1,\dots,d\}^{n}$ is the zero set of a non-negative polynomial of highest order (6.21), 6.22 implies the following.

Corollary 6.26**.**

Let $n,d\in\mathds{N}$, $\mathcal{X}=\mathds{R}^{n}$, and $\mathcal{A}=\mathds{R}[x_{1},\dots,x_{n}]_{\leq 2d}$. Then there is a moment sequence $s\in\mathrm{int}\,\mathcal{S}_{\mathsf{A}_{n,2d}}$ and an open neighborhood $U\subset\mathrm{int}\,\mathcal{S}_{\mathsf{A}_{n,2d}}$ of $s$ such that

[TABLE]

for all $s^{\prime}\in U$, i.e., every $s^{\prime}\in U$ is a linear combination of (33) many Gaussian distributions but not less.

Hence, like in [dDK19, Thm. 5.6] we have

[TABLE]

We end with the following asymptotic result which, as in the case of atomic measures [dDK19, Cor. 5.8], demonstrates that also the truncated moment problem with Gaussian mixtures is cursed by high dimensions. Note, an upper bound for the number of components is $\mathcal{C}^{M}_{\mathsf{A}_{n,2d}}\leq\binom{2d+n}{n}-1$, see [dD19, Thm. 32].

Corollary 6.27**.**

Let $d\in\mathds{N}$ and $\varepsilon>0$. Then there is an $n\in\mathds{N}$ such that there is a moment functional $L:\mathds{R}[x_{1},\dots,x_{n}]_{\leq 2d}\to\mathds{R}$ which can be written as a sum of

[TABLE]

Gaussian distributions but not less.

Acknowledgment

We thank Mario Kummer for the productive discussions and advise on the paper. We thank Mioara Joldes, Florent Bréhard, and Jean-Bernard Lasserre to provide the references [LPHT08, HK14, MWHL18, BJL19]. We want to thank Bernard Mourrain for the fruitful discussion at the Arctic Applied Algebra conference organized by Philippe Moustrou, Verena Reichle, Cordian Riener, and Hugues Verdure in Tromsø, April 2019.