First order covariance inequalities via Stein's method

TL;DR

This paper develops new covariance inequalities and Stein kernel expressions using Stein's method, providing explicit bounds and identities for univariate distributions with broad applications.

Contribution

It introduces probabilistic representations for inverse Stein operators, leading to new covariance identities and bounds that generalize classical variance inequalities.

Findings

Derived new simple expressions for Stein kernels.

Established sharp covariance bounds using weighted identities.

Connected results to classical variance bounds and recent literature.

Abstract

We propose probabilistic representations for inverse Stein operators (i.e. solutions to Stein equations) under general conditions; in particular we deduce new simple expressions for the Stein kernel. These representations allow to deduce uniform and non-uniform Stein factors (i.e. bounds on solutions to Stein equations) and lead to new covariance identities expressing the covariance between arbitrary functionals of an arbitrary {univariate} target in terms of a weighted covariance of the derivatives of the functionals. Our weights are explicit, easily computable in most cases, and expressed in terms of objects familiar within the context of Stein's method. Applications of the Cauchy-Schwarz inequality to these weighted covariance identities lead to sharp upper and lower covariance bounds and, in particular, weighted Poincar\'e inequalities. Many examples are given and, in particular, classical variance bounds due to Klaassen, Brascamp and Lieb or Otto and Menz are corollaries. Connections with more recent literature are also detailed.

Tables

Table 1. Table 1 : Specific forms some discrete distributions from the cumulative Ord family. This table is a completed version of Table 1 of [ 2 ] .

name	p.m.f. $p\left(x\right)$	Stein kernel ${\tau }^{\ell }\left(x\right)$
parameter	support	Cum. Ord relation
Poisson $\left(\lambda \right)$	$e^{-\lambda }{\lambda }^x/x!$	${\tau }^{-}\left(x\right)=\lambda$
$\lambda >0$	$x=0,1,\dots$	${\tau }^{+}\left(x\right)=x$
		$(\delta ,\beta ,\gamma )=(0,0,\lambda )$
\hdashline[0.5pt/5pt]	Stein operators	${𝒜}_{\mathrm{Poi}\left(\lambda \right)}^{+}g\left(x\right)=\left(x-\lambda \right)g\left(x\right)-x{\Delta }^{-}g\left(x\right)$
		${𝒜}_{\mathrm{Poi}\left(\lambda \right)}^{-}g\left(x\right)=\left(x-\lambda \right)g\left(x\right)-\lambda {\Delta }^{+}g\left(x\right)$
		${𝒜}_{\mathrm{Poi}\left(\lambda \right)}g\left(x\right)=xg\left(x\right)-\lambda g\left(x+1\right)$
\hdashline[0.5pt/5pt]	Variance bounds	$\lambda 𝔼{\left[{\Delta }^{+}g\left(X\right)\right]}^2\le \mathrm{Var}\left[g\left(X\right)\right]\le \lambda 𝔼\left[{\left({\Delta }^{+}g\left(X\right)\right)}^2\right]$
		${\lambda }^{-1}𝔼{\left[X{\Delta }^{-}g\left(X\right)\right]}^2\le \mathrm{Var}\left[g\left(X\right)\right]\le 𝔼\left[X{\left({\Delta }^{-}g\left(X\right)\right)}^2\right]$
Binomial $(n,\theta )$	$\left(\frac{n}{x}\right){\theta }^x{\left(1-\theta \right)}^{n-x}$	${\tau }^{-}\left(x\right)=\theta \left(n-x\right)$
	$x=0,1,\dots ,n$	${\tau }^{+}\left(x\right)=\left(1-\theta \right)x$
$n=1,2,\dots$		$(\delta ,\beta ,\gamma )=(0,-\theta ,n\theta )$
\hdashline[0.5pt/5pt]	Stein operators	${𝒜}_{\mathrm{bin}(n,\theta )}^{+}g\left(x\right)=\left(x-n\theta \right)g\left(x\right)-\left(1-\theta \right)x{\Delta }^{-}g\left(x\right)$
		${𝒜}_{\mathrm{bin}(n,\theta )}^{-}g\left(x\right)=\left(x-n\theta \right)g\left(x\right)-\theta \left(n-x\right){\Delta }^{+}g\left(x\right)$
		${𝒜}_{\mathrm{bin}(n,\theta )}g\left(x\right)=xg\left(x\right)+\frac{\theta }{1-\theta }\left(n-x\right)g\left(x+1\right)$
\hdashline[0.5pt/5pt]	Variance bounds	$\frac{\theta }{n\left(1-\theta \right)}𝔼{\left[\left(n-X\right){\Delta }^{+}g\left(X\right)\right]}^2\le \mathrm{Var}\left[g\left(X\right)\right]\le \theta 𝔼\left[\left(n-X\right){\left({\Delta }^{+}g\left(X\right)\right)}^2\right]$
		$\frac{1-\theta }{n\theta }𝔼{\left[X{\Delta }^{-}g\left(X\right)\right]}^2\le \mathrm{Var}\left[g\left(X\right)\right]\le \left(1-\theta \right)𝔼\left[X{\left({\Delta }^{-}g\left(X\right)\right)}^2\right]$
Negative Binomial $(r,p)$	$\left(\frac{r+x-1}{x}\right)p^r{\left(1-p\right)}^x$	${\tau }^{-}\left(x\right)=\frac{1-p}{p}\left(r+x\right)$
	$x=0,1,\dots$	${\tau }^{+}\left(x\right)=\frac{1}{p}x$
$r>0$		$(\delta ,\beta ,\gamma )=(0,\frac{1-p}{p},r\frac{1-p}{p})$
\hdashline[0.5pt/5pt]	Stein operators	${𝒜}_{\mathrm{NB}(r,p)}^{+}g\left(x\right)=\left(x-\frac{1-p}{p}r\right)g\left(x\right)-\frac{x}{p}{\Delta }^{-}g\left(x\right)$
		${𝒜}_{\mathrm{NB}(r,p)}^{-}g\left(x\right)=\left(x-\frac{1-p}{p}r\right)g\left(x\right)-\frac{1-p}{p}\left(r+x\right){\Delta }^{+}g\left(x\right)$
		${𝒜}_{\mathrm{NB}(r,p)}g\left(x\right)=xg\left(x\right)-\left(1-p\right)\left(r+x\right)g\left(x+1\right)$
\hdashline[0.5pt/5pt]	Variance bounds	$\frac{1-p}{r}𝔼{\left[\left(X+r\right){\Delta }^{+}g\left(X\right)\right]}^2\le \mathrm{Var}\left[g\left(X\right)\right]\le \frac{1-p}{p}𝔼\left[\left(X+r\right){\left({\Delta }^{+}g\left(X\right)\right)}^2\right]$
		$\frac{1}{r\left(1-p\right)}𝔼{\left[X{\Delta }^{-}g\left(X\right)\right]}^2\le \mathrm{Var}\left[g\left(X\right)\right]\le \frac{1}{p}𝔼\left[X{\left({\Delta }^{-}g\left(X\right)\right)}^2\right]$

Table 2. Table 2 : Specific form for some discrete distributions from the cumulative Ord family (second part).

name	p.m.f. $p\left(x\right)$	Stein kernel ${\tau }^{\ell }\left(x\right)$
parameter	support	Cum. Ord relation
Hypergeometric	$\frac{\left(\frac{K}{x}\right)\left(\frac{N-K}{n-x}\right)}{\left(\frac{N}{n}\right)}$	${\tau }^{-}\left(x\right)=\frac{1}{N}\left(K-x\right)\left(n-x\right)$
$(n,K,N)$	$x=0,1,\dots ,\min \{K,n\}$	${\tau }^{+}\left(x\right)=\frac{1}{N}x\left(x+N-K-n\right)$
$1\le K\le N$		$(\delta ,\beta ,\gamma )=(\frac{1}{N},\frac{-\left(n+K\right)}{N},\frac{nK}{N})$
$n=1,2,\dots ,N$
\hdashline[0.5pt/5pt]	Stein operators	${𝒜}_{H(n,K,N)}^{+}g\left(x\right)=\left(x-\frac{nK}{N}\right)g\left(x\right)-\frac{1}{N}x\left(N-K-n+x\right){\Delta }^{-}g\left(x\right)$
		${𝒜}_{H(n,K,N)}^{-}g\left(x\right)=\left(x-\frac{nK}{N}\right)g\left(x\right)-\frac{1}{N}\left(K-x\right)\left(n-x\right){\Delta }^{+}g\left(x\right)$
		${𝒜}_{H(n,K,N)}g\left(x\right)=xg\left(x\right)+\frac{1}{N+K+n}{x}^{2}g\left(x\right)-\frac{1}{N+K+n}\left(K-x\right)\left(n-x\right)g\left(x+1\right)$
\hdashline[0.5pt/5pt]	Variance bounds	$\frac{N-1}{nK\left(N-K\right)\left(N-n\right)}𝔼{\left[\left(K-X\right)\left(n-X\right){\Delta }^{+}g\left(X\right)\right]}^2\le \mathrm{Var}\left[g\left(X\right)\right]\le \frac{1}{N}𝔼\left[\left(K-X\right)\left(n-X\right){\left({\Delta }^{+}g\left(X\right)\right)}^2\right]$
		$\frac{N-1}{nK\left(N-K\right)\left(N-n\right)}𝔼{\left[X\left(N-K-n+X\right){\Delta }^{-}g\left(X\right)\right]}^2\le \mathrm{Var}\left[g\left(X\right)\right]\le \frac{1}{N}𝔼\left[X\left(N-K-n+X\right){\left({\Delta }^{-}g\left(X\right)\right)}^2\right]$
Negative Hyper-	$\frac{\left(\frac{x+r-1}{x}\right)\left(\frac{N-r-x}{K-x}\right)}{\left(\frac{N}{K}\right)}$	${\tau }^{-}\left(x\right)=\frac{1}{N-K+1}\left(K-x\right)\left(r+x\right)$
geometric $(N,K,r)$	$x=0,1,\dots ,K$	${\tau }^{+}\left(x\right)=\frac{1}{N-K+1}x\left(N-r+1-x\right)$
$0\le K\le N$		$(\delta ,\beta ,\gamma )=(\frac{-1}{N-K+1},\frac{K-r}{N-K+1},\frac{rK}{N-K+1})$
\hdashline[0.5pt/5pt]	Stein operators	${𝒜}_{\mathrm{NH}(N,K,r)}^{+}g\left(x\right)=\left(x-\frac{rK}{N-K+1}\right)g\left(x\right)-\frac{x\left(N+1-r-x\right)}{N-K+1}{\Delta }^{-}g\left(x\right)$
		${𝒜}_{\mathrm{NH}(N,K,r)}^{-}g\left(x\right)=\left(x-\frac{rK}{N-K+1}\right)g\left(x\right)-\frac{\left(K-x\right)\left(r+x\right)}{N-K+1}{\Delta }^{+}g\left(x\right)$
		${𝒜}_{\mathrm{NH}(N,K,r)}g\left(x\right)=xg\left(x\right)-\frac{1}{N-r+1}{x}^{2}g\left(x\right)-\frac{1}{N-r+1}\left(K-x\right)\left(r+x\right)g\left(x+1\right)$
\hdashline[0.5pt/5pt]	Variance bounds	$\frac{N-K+2}{r\left(N+1\right)K\left(N-K-r+1\right)}𝔼{\left[\left(K-X\right)\left(r+X\right){\Delta }^{+}g\left(X\right)\right]}^2\le \mathrm{Var}\left[g\left(X\right)\right]$
		$\mathrm{Var}\left[g\left(X\right)\right]\le \frac{1}{N-K+1}𝔼\left[\left(K-X\right)\left(r+X\right){\left({\Delta }^{+}g\left(X\right)\right)}^2\right]$
		$\frac{N-K+2}{r\left(N+1\right)K\left(N-K-r+1\right)}𝔼{\left[X\left(N+1-r-X\right){\Delta }^{-}g\left(X\right)\right]}^2\le \mathrm{Var}\left[g\left(X\right)\right]$
		$\mathrm{Var}\left[g\left(X\right)\right]\le \frac{1}{N-K+1}𝔼\left[X\left(N+1-r-X\right){\left({\Delta }^{-}g\left(X\right)\right)}^2\right]$

Table 3. Table 3 : Specific form for some continuous distributions. If the distribution belongs to the Pearson family (examples from [ 1 ] ), the coefficient ( δ , β , γ ) 𝛿 𝛽 𝛾 (\delta,\beta,\gamma) are given. This table is an adapted version of Table 1 of [ 2 ] .

name	p.m.f. $p\left(x\right)$	$\tau \left(x\right)$
parameter	support	Pearson relation
Normal$(\mu ,{\sigma }^2)$	$\frac{1}{\sqrt{2\pi {\sigma }^2}}\exp \left(\frac{{\left(x-\mu \right)}^2}{2{\sigma }^2}\right)$	$\tau \left(x\right)={\sigma }^2$
$\mu \in \mathrm{IR}$, ${\sigma }^2>0$	$x\in \mathrm{IR}$	$(\delta ,\beta ,\gamma )=(0,0,{\sigma }^2)$
\hdashline[0.5pt/5pt]	Stein operators	${𝒜}_{N(\mu ,{\sigma }^2)}g\left(x\right)=\left(x-\mu \right)g\left(x\right)-{\sigma }^2{g}^{\prime }\left(x\right)$
\hdashline[0.5pt/5pt]	Variance bounds	${\sigma }^2𝔼{\left[g^{\prime }\left(X\right)\right]}^2\le \mathrm{Var}\left[f\left(X\right)\right]\le {\sigma }^2𝔼\left[g^{\prime }{\left(X\right)}^2\right]$
Beta $(\alpha ,\beta )$	$x^{\alpha -1}{\left(1-x\right)}^{\beta -1}/B(\alpha ,\beta )$	$\tau \left(x\right)=\frac{x\left(1-x\right)}{\alpha +\beta }$
$\alpha >0$, $\beta >0$	$x\in (0,1)$	$(\delta ,\beta ,\gamma )=(\frac{-1}{\alpha +\beta },\frac{1}{\alpha +\beta },0)$
\hdashline[0.5pt/5pt]	Stein operators	${𝒜}_{\mathrm{Beta}(\alpha ,\beta )}g\left(x\right)=\left(x-\frac{\alpha }{\alpha +\beta }\right)g\left(x\right)-\frac{x\left(1-x\right)}{\alpha +\beta }{g}^{\prime }\left(x\right)$
\hdashline[0.5pt/5pt]	Variance bounds	$\frac{\left(\alpha +\beta +1\right)}{\alpha \beta }𝔼{\left[X\left(1-X\right)g^{\prime }\left(X\right)\right]}^2\le \mathrm{Var}\left[g\left(X\right)\right]\le \frac{1}{\alpha +\beta }𝔼\left[X\left(1-X\right){\left(g^{\prime }\left(X\right)\right)}^2\right]$
Gamma$(\mu ,{\sigma }^2)$	$x^{\alpha -1}{\beta }^{-\alpha }{e}^{-x/\beta }/\Gamma \left(\alpha \right)$	$\tau \left(x\right)=\beta x$
$\alpha >0$, $\beta >0$		$(\delta ,\beta ,\gamma )=(0,\beta ,0)$
	$x\in [0,\infty )\left(\alpha \ge 1\right)$
\hdashline[0.5pt/5pt]	Stein operators	${𝒜}_{\mathrm{Gamma}(\alpha ,\beta )}g\left(x\right)=\left(x-\alpha \beta \right)g\left(x\right)-\beta x{g}^{\prime }\left(x\right)$
\hdashline[0.5pt/5pt]	Variance bounds	$\frac{1}{\alpha }𝔼{\left[X{g}^{\prime }\left(X\right)\right]}^2\le \mathrm{Var}\left[g\left(X\right)\right]\le \beta 𝔼\left[X{g}^{\prime }{\left(X\right)}^2\right]$
Student $\left(\nu \right)$	${\nu }^{-1/2}B{(\nu /2,1/2)}^{-1}{\left(\nu /\left(\nu +x^2\right)\right)}^{\left(1+\nu \right)/2}$	$\tau \left(x\right)=\frac{x^2+\nu }{\nu -1}$ for $\nu >1$
$\nu >0$	$x\in \mathrm{IR}$	$(\delta ,\beta ,\gamma )=(\frac{1}{\nu -1},0,\frac{\nu }{\nu -1})$
\hdashline[0.5pt/5pt]	Stein operators	${𝒜}_{t\left(\nu \right)}g\left(x\right)=xg\left(x\right)-\frac{x^2+\nu }{\nu -1}{g}^{\prime }\left(x\right)$
\hdashline[0.5pt/5pt]	Variance bounds ($\nu >2$)	$\frac{\left(\nu -2\right)}{\nu {\left(\nu -1\right)}^2}𝔼{\left[\left(X^2+\nu \right)g^{\prime }\left(X\right)\right]}^2\le \mathrm{Var}\left[g\left(X\right)\right]\le \frac{1}{\nu -1}𝔼\left[\left(X^2+\nu \right)\left(g^{\prime }{\left(X\right)}^2\right)\right]$
F distribution $(d_1,d_2)$	$\frac{{\left(\frac{d_1}{d_2}\right)}^{d_1/2}{x}^{\frac{d_1}{2}-1}{\left(1+\frac{d_1}{d_2}x\right)}^{-\frac{d_1+d_2}{2}}}{\text{Beta}(d_1/2,d_2/2)}$	$\tau \left(x\right)=\frac{2x\left(d_{1}x+d_2\right)}{d_1\left(d_2-2\right)}$ for $d_2>2$
$d_1>0$, $d_2>0$	$x\in (0,\infty )$	$(\delta ,\beta ,\gamma )=(\frac{2d_1}{d_1\left(d_2-2\right)},\frac{2d_2}{d_1\left(d_2-2\right)},0)$
\hdashline[0.5pt/5pt]	Stein operators	${𝒜}_{F(d_1,d_2)}g\left(x\right)=\left(x-\frac{d_2}{d_2-1}\right)g\left(x\right)-\frac{2x(d_2+d_{1}x}{d1\left(d_2-2\right)}{g}^{\prime }\left(x\right)$
\hdashline[0.5pt/5pt]	Variance bounds ($d_2>4$)	$\frac{2\left(d_2-4\right)}{d_1{d}_2^2\left(d_1+d_2-2\right)}𝔼{\left[X\left(d_2+d_{1}X\right)g^{\prime }\left(X\right)\right]}^2\le \mathrm{Var}\left[g\left(X\right)\right]\le \frac{2}{d_1\left(d_2-2\right)}𝔼\left[X\left(d_2+d_{1}X\right)g^{\prime }{\left(X\right)}^2\right]$

Equations

$$\mathbb{E}[g^{\prime }(N){]}^2\le \mathrm{Var}[g(N)]\le \mathbb{E}[g^{\prime }(N{)}^2]$$

$$\frac{\mathbb{E}[{\tau }_p(X)g^{\prime }(X){]}^2}{\mathrm{Var}[X]}\le \mathrm{Var}[g(X)]\le \mathbb{E}\left[{\tau }_p(X)g^{\prime }(X{)}^2\right]$$

$$\mathrm{Var}[g(X)]\le \mathbb{E}\left[\frac{(g^{\prime }(X){)}^2}{(-\log p{)}^{\prime \prime }(X)}\right]$$

$$\mathrm{Cov}[f(X),g(Y)]=\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}f^{\prime}(x)\Big{(}H(x,y)-F(x)G(y)\Big{)}g^{\prime}(y)\,\mathrm{d}x\,\mathrm{d}y$$

$$\left|\mathrm{Cov}[f(X),g(X)]\right|\leq\sup_{x}\left\{\frac{|f^{\prime}(x)|}{(\log p)^{\prime\prime}(x)}\right\}\mathbb{E}\big{[}\left|g^{\prime}(X)\right|\big{]}.$$

$$\mathrm{Var}[G(X)]\le \mathbb{E}\left[\frac{g(X{)}^2}{h(X)}\left(\frac{1}{p(X)}\int \rho (z,X)H(z)p(z)\mu (\mathrm{d}z)\right)\right]$$

$$\mathrm{Var}[G(X)]\ge \frac{\mathbb{E}{\left[g(X)K(X)\right]}^2}{\mathrm{Var}[k(X)]}$$

$$\mathbb{E}[Ng(N)]=\mathbb{E}[g^{\prime }(N)]\text{mbox}forallg\text{mbox}suchthat\mathbb{E}[|g^{\prime }(N)|]<\infty .$$

$$\mathbb{E}[g^{\prime }(N){]}^2=\mathbb{E}[Ng(N){]}^2\le \mathbb{E}[N^2]\mathbb{E}\left[g(N{)}^2\right]\le \mathrm{Var}[g(N)],$$

$$\mathrm{Var}[g(N)]\le \mathbb{E}\left[{\left({\int }_0^N{g}^{\prime }(x)\mathrm{d}x\right)}^2\right]$$

$${\Delta }^{\ell }f(x)$$

$${\int }_c^d{\Delta }^{\ell }f(x)\mu (\mathrm{d}x)=\{\begin{array}{ll}{\int }_c^d{f}^{\prime }(x)\mathrm{d}x=f(d)-f(c)& \text{mbox}if\ell =0\\ {\sum }_{j=c}^d{\Delta }^{-}f(x)=f(d)-f(c-1)& \text{mbox}if\ell =-1\\ {\sum }_{j=c}^d{\Delta }^{+}f(x)=f(d+1)-f(c)& \text{mbox}if\ell =+1\end{array}$$

$${\int }_c^d{\Delta }^{\ell }f(x)\mu (\mathrm{d}x)=f(d+a_{\ell })-f(c-b_{\ell })$$

$$a_{\ell }=\mathbb{I}[\ell =1]\text{mbox}and{b}_{\ell }=\mathbb{I}[\ell =-1].$$

$${\mathcal{T}}_p^{\ell }f(x)=\frac{{\Delta }^{\ell }(f(x)p(x))}{p(x)}$$

$${\mathcal{F}}^{(0)}(p)$$

$${\mathcal{F}}_{\ell }^{(1)}(p)$$

$$f(b+a_{\ell })p(b+a_{\ell })=f(a-b_{\ell })p(a-b_{\ell }).$$

$$\Delta^{\ell}\big{(}f(x)g(x-\ell)\big{)}=(\Delta^{\ell}f(x))g(x)+f(x)\Delta^{-\ell}g(x)$$

$$\mathbb{E}\left[({\mathcal{T}}_p^{\ell }f(X))g(X)\right]=-\mathbb{E}\left[f(X){\Delta }^{-\ell }g(X)\right].$$

$$\mathcal{T}_{p}^{\ell}\big{(}f(x)g(x-\ell)\big{)}=(\mathcal{T}_{p}^{\ell}f(x))g(x)+f(x)(\Delta^{-\ell}g(x))$$

$${\mathcal{L}}_p^{\ell }h(x)=\frac{1}{p(x)}{\int }_a^{x-a_{\ell }}(h(u)-\mathbb{E}[h(X)])p(u)\mu (\mathrm{d}u)=\frac{1}{p(x)}{\int }_{x+b_{\ell }}^b(\mathbb{E}[h(X)]-h(u))p(u)\mu (\mathrm{d}u)$$

$${\mathcal{L}}_p^{0}h(x)=\frac{1}{p(x)}{\int }_a^x(h(u)-\mathbb{E}[h(X)])p(u)\mathrm{d}u=\frac{1}{p(x)}{\int }_x^b(\mathbb{E}[h(X)]-h(u))p(u)\mathrm{d}u$$

$${\mathcal{L}}_p^{-}h(x)=\frac{1}{p(x)}\sum _{j=a}^x(h(j)-\mathbb{E}[h(X)])p(j)=\frac{1}{p(x)}\sum _{j=x+1}^b(\mathbb{E}[h(X)]-h(j))p(j)$$

$${\mathcal{L}}_p^{+}h(x)=\frac{1}{p(x)}\sum _{j=a}^{x-1}(h(j)-\mathbb{E}[h(X)])p(j)=\frac{1}{p(x)}\sum _{j=x}^b(\mathbb{E}[h(X)]-h(j))p(j)$$

$${\mathcal{A}}_p^{\ell ,\eta }g(x)$$

$$\big{(}\eta(x)-\mathbb{E}[\eta(X)]\big{)}g(x)+\mathcal{L}^{\ell}_{p}\eta(x)\big{(}\Delta^{-\ell}g(x)\big{)}=h(x)-\mathbb{E}[h(X)],\,x\in\mathcal{S}(p).$$

$$g_h^{p,\ell ,\eta }(x)=\frac{{\mathcal{L}}_p^{\ell }h(x+\ell )}{{\mathcal{L}}_p^{\ell }\eta (x+\ell )}$$

$${\mathcal{A}}_p^{\ell ,\eta }g(x)$$

$$=h(x)-\mathbb{E}[h(X)]$$

$${\mathcal{A}}_{\mathrm{bin}(n,\theta )}^{+,\mathrm{Id}}g(x)=(x-n\theta )g(x)-(1-\theta )x{\Delta }^{-}g(x)$$

$$g^{+}(x)=\frac{-1}{(1-\theta )(x+1)p(x+1)}\sum _{j=0}^x(h(j)-\mathbb{E}[h(X)])p(j)\text{mbox}forall0\le x\le n-1$$

Full text

1 Introduction

Much attention has been given in the literature to the problem of providing sharp tractable estimates on the variance of functions of random variables. Such estimates are directly related to fundamental considerations of pure mathematics (e.g., isoperimetric, logarithmic Sobolev and Poincaré inequalities), as well as essential issues from statistics (e.g., Cramer-Rao bounds, efficiency and asymptotic relative efficiency computations, maximum correlation coefficients, and concentration inequalities).

One of the starting points of this line of research is Chernoff’s famous result from [27] which states that, if $N\sim\mathcal{N}(0,1)$, then

[TABLE]

for all sufficiently regular functions $g:{\rm I\kern-1.60004ptR}\to{\rm I\kern-1.60004ptR}$. Chernoff obtained the upper bound by exploiting orthogonality properties of the family of Hermite polynomials. The upper bound in (1.1) is, in fact, already available in [56] and is also a special case of the central inequality in [11], see below. Cacoullos [12] extends Chernoff’s bound to a wide class of univariate distributions (including discrete distributions) by proving that if $X\sim p$ has a density function $p$ with respect to the Lebesgue measure then

[TABLE]

with $\tau_{p}(x)={p(x)}^{-1}\int_{x}^{\infty}(t-\mathbb{E}[X])p(t)dt$. It is easy to see that, if $p$ is the standard normal density, then $\tau_{p}(x)=1$ so that (1.2) contain (1.1). Cacoullos also obtains a similar bound as (1.2) for discrete distributions on the positive integers, where the derivative is replaced by the forward difference and the weight becomes $\tau_{p}(x)={p(x)}^{-1}\sum_{t=x+1}^{\infty}tp(t).$

Variance inequalities such as (1.2) are closely related to the celebrated Brascamp-Lieb inequality from [11] which, in dimension 1, states that if $X\sim p$ and $p$ is strictly log-concave then

[TABLE]

for all sufficiently regular functions $g$. In fact, the upper bound from (1.1) is an immediate consequence of (1.3) because, if $p$ is the standard Gaussian density, then $(-\log p)^{\prime\prime}(x)\equiv 1$. The Brascamp-Lieb inequality is proved in [55] to be a consequence of Hoeffding’s classical covariance inequality from [41], which states that if $(X,Y)$ is a continuous bivariate random vector with cumulative distribution $H(x,y)$ and marginal cdfs $F(x),G(x)$ then

[TABLE]

under weak assumptions on $f,g$ (see e.g. [29]). The freedom of choice in the test functions $f,g$ in (1.4) is exploited by [55] to prove that, if $X$ has a $C^{2}$ strictly convex absolutely continuous density $p$ then the asymmetric Brascamp-Lieb inequality holds:

[TABLE]

Identity (1.4) and inequalities (1.3) and (1.5) are extended to the multivariate setting in [19] which also gives connections with logarithmic Sobolev inequalities for spin systems and related inequalities for log-concave densities. This material is revisited and extended in [66, 64, 65], providing applications in the context of isoperimetric inequalities and weighted Poincaré inequalities. In [29] the identity (1.4) is proved in all generality and used to provide expansions for the covariance in terms of canonical correlations and variables.

Further generalizations of Chernoff’s bounds are provided in [24, 14, 15], and [44] (e.g., Karlin [44] deals with the entire class of log-concave distributions). See also [10, 16, 46, 58, 18] for the connection with probabilistic characterizations and other properties. Similar inequalities were obtained – often by exploiting properties of suitable families of orthogonal polynomials – for univariate functionals of some specific multivariate distributions e.g., in [17, 13, 20, 48, 3, 49]. A historical overview as well as a description of the connection between such bounds, the so-called Stein identities from Stein’s method (see below) and Sturm-Liouville theory (see Section 4) can be found in [30]. To the best of our knowledge, the most general version of (1.1) and (1.2) is due to [45], where the following result is proved

Theorem 1.1** (Klaassen bounds).**

Let $\mu$ be some $\sigma$-finite measure. Let ${\rho}(x,y)$ be a measurable function such that ${\rho}(x,\cdot)$ does not change sign for $\mu$ almost $x\in\mathbb{R}$. Suppose that $g$ is a measurable function such that $G(x)=\int{\rho}(x,y)g(y)\,\mu(\mathrm{d}y)+c$ is well defined for some $c\in{\rm I\kern-1.60004ptR}$. Let $X$ be a real random variable with density $p$ with respect to $\mu$.

• •

(Klaassen upper variance bound) For all nonnegative measurable functions $h:{\rm I\kern-1.60004ptR}\to{\rm I\kern-1.60004ptR}$ such that $\mu\left(\left\{x\in{\rm I\kern-1.60004ptR}\,|\,g(x)\neq 0,\,p(x)h(x)=0\right\}\right)=0$ we have

[TABLE]

with $H:{\rm I\kern-1.60004ptR}\to{\rm I\kern-1.60004ptR}$ supposed well-defined by $H(x)=\int{\rho}(x,y)h(y)\,\mu(\mathrm{d}y)$.

• •

(Cramér-Rao lower variance bound) For all measurable functions $k:{\rm I\kern-1.60004ptR}\to{\rm I\kern-1.60004ptR}$ such that $0<\mathbb{E}[k^{2}(X)]<\infty$ and $\mathbb{E}[k(X)]=0$ we have

[TABLE]

where $K(x)=\frac{1}{p(x)}\int{\rho}(z,x)k(z)p(z)\mu(\mathrm{d}z)$. Equality in (1.7) holds if and only if $G$ is linear in $k$, $p$-almost everywhere.

Klaassen’s proof of Theorem 1.1 relies on little more than the Cauchy-Schwarz inequality and Fubini’s theorem; it has a slightly magical aura as little or no heuristic or context is provided as to the best choices of test functions $h,k$ and kernel $\rho$ or even to the nature of the weights appearing in (1.6) and (1.7). To the best of our knowledge, all available first order variance bounds from the literature can be obtained from either (1.6) or (1.7) by choosing the appropriate test functions $h$ or $k$ and the appropriate kernel $\rho$. For instance, the weights appearing in the upper bound (1.6) generalize the Stein kernel from Cacoullos’ bound (1.2) – both in the discrete and the continuous case. Indeed taking $H(x)=x$ when the distribution $p$ is continuous we see that then $h(x)=1$ and the weight becomes $p(x)^{-1}\int\rho(z,x)zp(z)\mathrm{d}\mu(z)$ which is none other than $\tau_{p}(x)$. A similar argument holds as well in the discrete case. In the same way, taking $k(x)=x$ leads to $K(x)=\tau_{p}(x)$ in (1.7) and thus the lower bound in (1.2) follows as well. The freedom of choice in the function $h$ allows for much flexibility in the quality of the weights; this fact seems somewhat under exploited in the literature. This is perhaps due to the rather obscure nature of Klaassen’s weights, a topic which we shall be one of the central learnings of this paper. Indeed we shall provide a natural theoretical home for Klaassen’s result, in the framework of Stein’s method.

Several variations on Klaassen’s theorem have already been obtained via techniques related to Stein’s method. We defer a proper introduction of these techniques to Section 2. The gist of the approach can nevertheless be understood very simply in case the underlying distribution is standard normal. Stein’s classical identity states that if $N\sim\mathcal{N}(0,1)$ then

[TABLE]

By the Cauchy-Schwarz inequality we immediately deduce that, for all appropriate $g$,

[TABLE]

which gives the lower bound in (1.1). For the upper bound, still by the Cauchy-Schwarz inequality,

[TABLE]

where the last identity is a direct consequence of Stein’s identity (1.8) applied to the function $g(x)=\int_{0}^{x}(g^{\prime}(u))^{2}\mathrm{d}u$. This is the upper bound in (1.1). The idea behind this proof is due to Chen [23]. As is now well known (again, we refer the reader to Section 2 for references and details), Stein’s identity (1.8) for the normal distribution can be extended to basically any univariate (and even multivariate) distribution via a family of objects called “Stein operators”. This leads to a wide variety of Stein-type integration by parts identities and it is natural to wonder whether Chen’s approach can be used to obtain generalizations of Klaassen’s theorem. First steps in this direction are detailed in [51, 52]; in particular it is seen that general lower variance bounds are easy to obtain from generalized Stein identities in the same way as in (1.9). Nevertheless, the method of proof in (1.10) for the upper bound cannot be generalized to arbitrary targets and, even in cases where the method does apply, the assumptions under which the bounds hold are quite stringent. To the best of our knowledge, the first to obtain upper variance bounds via properties of Stein operators is due to Saumard [64], by combining generalized Stein identities – expressed in terms of the Stein kernel $\tau_{p}(x)$ – with Hoeffding’s identity (1.4). The scope of Saumard’s weighted Poincaré inequalities is, nevertheless, limited and a general result such as Klaassen’s is, to this date, not available in the literature.

The main contributions of this paper can be categorized in two types:

• •

Covariance identities and inequalities. The first main contribution of this paper is a generalization of Klaassen’s variance bounds from Theorem 1.1 to covariance inequalities of arbitrary functionals of arbitrary univariate targets under minimal assumptions (see Theorems 3.1 and 3.5). Our results hereby therefore also contains basically the entire literature on the topic. Moreover, the weights that appear in our bounds bear a clear and natural interpretation in terms of Stein operators which allow for easy computation for a wide variety of targets, as illustrated in the different examples we tackle as well as in Tables 1, 2 and 3 in which we provide explicit variance bounds for univariate target distributions belonging to the classical integrated Pearson and Ord families (see Example 3.8 for a definition). In particular, Klaassen’s bounds now arise naturally in this setting.

• •

Stein operators and their properties. The second main contribution of the paper lies in our method of proof, which contributes to the theory of Stein operators themselves. Specifically, we obtain several new probabilistic representations of inverse Stein operators (a.k.a. solutions to Stein equations) which open the way to a wealth of new manipulations which where hitherto unavailable. These representations also lead to new interpretations and ultimately new handles on several quantities which are crucial to the theory surrounding Stein’s method (such as Stein kernels, Stein equations, Stein factors, and Stein bounds). Finally the various objects we identify provide natural connections with other topics of interest, including the well-known connection with Sturm-Liouville theory already identified in [30].

The paper is organised as follows. Section 2 contains the theoretical foundations of the paper. In Section 2.1 we recall the theory of canonical and standardized Stein operators introduced in [53] and introduce a (new) notion of inverse Stein operator (Definition 2.4). We also identify minimal conditions under which Stein-type probabilistic integration by parts formulas hold (see Lemmas 2.3 and 2.16). In Section 2.2 we provide the representation formulas for the inverse Stein operator (Lemmas 2.18 and 2.19). In Section 2.3 we clarify the conditions on the test functions under which the different identities hold, and provide bridges with the classical assumptions in the literature. Section 2.4 contains bounds on the solutions to the Stein equations. Section 3 contains the covariance identities and inequalities. After re-interpreting Hoeffding’s identity (1.4) we obtain general and flexible lower and upper covariance bounds (Proposition 3.1 and Theorem 3.5). We then deduce Klaassen’s bounds (Corollary 3.7) and provide examples for several concrete distributions, with more examples deferred to the three tables mentioned above. Finally a discussion is provided in Section 4, wherein several examples are treated and connections with other theories are established, for instance the Brascamp-Lieb inequality (Corollary 4.1) and Menz and Otto’s asymmetric Brascamp-Lieb inequality (Corollary 4.2), as well as the link with an eigenfunction problem which can be seen as an extended Sturm-Liouville problem. The proofs from Section 2.3 are technical and postponed to the appendix A.

2 Stein differentiation

Stein’s method consists in a collection of techniques for distributional approximation that was originally developed for normal approximation in [69] and for Poisson approximation in [25]; for expositions see the books [70, 7, 8, 26, 57] and the review papers [61, 63, 21]. Outside the Gaussian and Poisson frameworks, there exist several non-equivalent general theories allowing to setup Stein’s method for large swaths of probability distributions, of which we single out the papers [22, 31, 71] for univariate distributions under analytical assumptions, [4, 5] for infinitely divisible distributions, [6] for discrete multivariate distributions, and [54, 38, 39] as well as [34] for multivariate densities under diffusive assumptions.

The backbone of the present paper consists in the approach from [50, 53, 62]. Before introducing these results, we fix the notations. Let $\mathcal{X}{\in\mathcal{B}({\rm I\kern-1.60004ptR})}$ and equip it with some $\sigma$-algebra $\mathcal{A}$ and $\sigma$-finite measure $\mu$. Let $X$ be a random variable on $\mathcal{X}$, with induced probability measure $\mathbb{P}^{X}$ which is absolutely continuous with respect to $\mu$; we denote by $p$ the corresponding probability density, and its support by $\mathcal{S}(p)=\left\{x\in\mathcal{X}:p(x)>0\right\}$. As usual, $L^{1}(p)$ is the collection of all real valued functions $f$ such that $\mathbb{E}|f(X)|<\infty$. We sometimes call the expectation under $p$ the $p$-mean. Although we could in principle keep the discussion to come very general, in order to make the paper more concrete and readable we shall restrict our attention to distributions satisfying the following Assumption.

Assumption A. The measure $\mu$ is either the counting measure on $\mathcal{X}={\mathbb{Z}}$ or the Lebesgue measure on $\mathcal{X}={\rm I\kern-1.60004ptR}$. If $\mu$ is the counting measure then there exist $a{<}b\in{\mathbb{Z}}\cup\left\{-\infty,\infty\right\}$ such that $\mathcal{S}(p)=[a,b]\cap{\mathbb{Z}}$. If $\mu$ is the Lebesgue measure then there exist $a,b\in{\rm I\kern-1.60004ptR}\left\{-\infty,\infty\right\}$ such that ${\mathcal{S}(p)}^{\mathrm{o}}=]a,b[$ and $\overline{\mathcal{S}(p)}=[a,b]$. Moreover, the measure $\mu$ is not point mass.

Here not allowing point mass much simplifies the presentation. Stein’s method for point mass is available in [60].

Let $\ell\in\left\{-1,0,1\right\}$. In the sequel we shall restrict our attention to the following three derivative-type operators:

[TABLE]

with $f^{\prime}(x)$ the weak derivative defined Lebesgue almost everywhere, $\Delta^{+1}({\equiv}\Delta^{+})$ the classical forward difference and $\Delta^{-1}({\equiv}\Delta^{-})$ the classical backward difference. Whenever $\ell=0$ we take $\mu$ as the Lebesgue measure and speak of the continuous case; whenever $\ell\in\left\{-1,1\right\}$ we take $\mu$ as the counting measure and speak of the discrete case. There are two choices of derivatives in the discrete case, only one in the continuous case. We let $\mathrm{dom}(\Delta^{\ell})$ denote the collection of functions $f:{\rm I\kern-1.60004ptR}\to{\rm I\kern-1.60004ptR}$ such that $\Delta^{\ell}f(x)$ exists and is finite $\mu$-almost surely. In the case $\ell=0$, this corresponds to all absolutely continuous functions; in the case $\ell=\pm 1$ the domain is the collection of all functions on ${\mathbb{Z}}$. For ease of reference we note that, if $f\in\mathrm{dom}(\Delta^{\ell})$ is such that $\Delta^{\ell}f\,\mathbb{I}[a,b]\in L^{1}(\mu)$ then, for all $c,d$ such that $a\leq c\leq d\leq b$ we have

[TABLE]

which we summarize as

[TABLE]

where

[TABLE]

We stress the fact that the values at $c,d$ are understood as limits if either is infinite.

2.1 Stein operators and Stein equations

Our first definitions come from [53]. We first define $\mathrm{dom}(p,\Delta^{\ell})$ as the collection of $f:{\rm I\kern-1.60004ptR}\to{\rm I\kern-1.60004ptR}$ such that $f\,p\in\mathrm{dom}(\Delta^{\ell})$.

Definition 2.1** (Canonical Stein operators).**

Let $f\in\mathrm{dom}(p,\Delta^{\ell})$ and consider the linear operator $f\mapsto\mathcal{T}_{p}^{\ell}f$ defined as

[TABLE]

for all $x\in\mathcal{S}(p)$ and $\mathcal{T}_{p}^{\ell}f(x)=0$ for $x\notin\mathcal{S}(p)$. The operator $\mathcal{T}_{p}^{\ell}$ is called the canonical ($\ell$-)Stein operator of $p$. The cases $\ell=1$ and $\ell=-1$ provide the forward and backward Stein operators, denoted by $\mathcal{T}_{p}^{+}$ and $\mathcal{T}_{p}^{-}$, respectively; the case $\ell=0$ provides the differential Stein operator denoted by $\mathcal{T}_{p}$.

To describe the domain and the range of $\mathcal{T}_{p}^{\ell}$ we introduce the following sets of functions:

[TABLE]

We draw the reader’s attention to the fact that the second condition in the definition of $\mathcal{F}^{(1)}_{\ell}(p)$ can be rewritten

[TABLE]

The next lemma, which follows immediately from the definition of $\mathcal{T}_{p}^{\ell}f$ and of the different sets of functions, shows why $\mathcal{F}^{(1)}_{\ell}(p)$ is called the canonical Stein class.

Lemma 2.2** (Canonical Stein class).**

For $f\in\mathcal{F}^{(1)}_{\ell}(p)$, $\mathcal{T}_{p}^{\ell}f\in\mathcal{F}^{(0)}(p)$ .

Crucially for the results in this paper, for all $f\in\mathrm{dom}(\Delta^{\ell})$, $g\in\mathrm{dom}(\Delta^{-\ell})$ such that $f(\cdot)g(\cdot-\ell)\in\mathrm{dom}(\Delta^{\ell})$ the operators $\Delta^{\ell}$ satisfy the product rule

[TABLE]

for all $\ell\in\left\{-1,0,1\right\}$. This product rule leads to an integration by parts (IBP) formula (a.k.a. Abel-type summation formula) as follows.

Lemma 2.3** (Stein IBP formula - version 1).**

For all $f\in\mathrm{dom}(p,\Delta^{\ell})$, $g\in\mathrm{dom}(\Delta^{-\ell})$ such that (i) $f(\cdot)g(\cdot-\ell)\in\mathcal{F}_{\ell}^{(1)}(p)$ and (ii) $f(\cdot)\Delta^{-\ell}g(\cdot)\in L^{1}(p)$ we have

[TABLE]

Proof.

Under the stated assumptions, we can apply (2.4) to get

[TABLE]

for all $x\in\mathcal{S}(p)$. Condition (i) in the statement guarantees that the left hand side (l.h.s.) of (2.6) has mean 0, while condition (ii) guarantees that we can separate the expectation of the sum on the right hand side (r.h.s.) into the sum of the individual expectations. ∎

A natural interpretation of (2.5) is that operator $\mathcal{T}_{p}^{\ell}$ is, in some sense to be made precise, the skew-adjoint operator to $\Delta^{-\ell}$ with respect to the scalar product $\left\langle f,g\right\rangle=\mathbb{E}\left[f(X)g(X)\right]$; this provides a supplementary justification to the use of the terminology “canonical” for operator $\mathcal{T}_{p}^{\ell}$. We discuss a consequence of this interpretation in Section 4. The conditions under which Lemma 2.3 holds are all but transparent. We clarify these assumptions in Section 2.3. For more details on Stein class and operators, we refer to [53] for the construction in an abstract setting, [50] for the construction in the continuous setting (i.e. $\ell=0$) and [33] for the construction in the discrete setting (i.e. $\ell\in\left\{-1,1\right\}$). Multivariate extensions are developed in [62].

The fundamental stepping stone for our theory is an inverse of the canonical operator $\mathcal{T}_{p}^{\ell}$ provided in the next definition.

Definition 2.4** **(Canonical pseudo inverse Stein

operator).

Let $\ell\in\left\{-1,0,1\right\}$ and recall the notations $a_{\ell},b_{\ell}$ from (2.3). The canonical pseudo-inverse Stein operator $\mathcal{L}_{p}^{\ell}$ for the operator $\mathcal{T}_{p}^{\ell}$ is defined, for $h\in L^{1}(p)$, as

[TABLE]

for all $x\in\mathcal{S}(p)$ and $\mathcal{L}_{p}^{\ell}h(x)=0$ for all $x\notin\mathcal{S}(p)$.

Equality between the second and third expressions in (2.7) is justified because $h\in L^{1}(p)$ so that the integral of $h(\cdot)-\mathbb{E}[h(X)]$ over the whole support cancels out. For ease of reference we detail $\mathcal{L}_{p}^{\ell}$ in the three cases that interest us:

[TABLE]

Note that $\mathcal{L}_{p}^{-}h(b)=0$ but $\mathcal{L}_{p}^{-}h(a)=h(a)-\mathbb{E}[h(X)]$ and, conversely, $\mathcal{L}_{p}^{+}h(a)=0$ but $\mathcal{L}_{p}^{+}h(b)=\mathbb{E}[h(X)]-h(b)$. The denomination pseudo-inverse-Stein operator for $\mathcal{L}_{p}^{\ell}$ is justified by the following lemma whose proof is immediate.

Lemma 2.5**.**

For any $h\in L^{1}(p)$, $\mathcal{L}^{\ell}_{p}h\in\mathcal{F}_{\ell}^{(1)}(p)$. Moreover, (i) for all $h\in L^{1}(p)$ we have $\mathcal{T}_{p}^{\ell}\mathcal{L}^{\ell}_{p}h(x)=h(x)-\mathbb{E}[h(X)]$ at all $x\in\mathcal{S}(p)$ and (ii) for all $f\in\mathcal{F}^{(1)}_{\ell}(p)$ we have $\mathcal{L}^{\ell}_{p}\mathcal{T}_{p}^{\ell}f(x)=f(x)-\frac{f(a^{+}-b_{\ell})p(a^{+}-b_{\ell})}{p(x)}=f(x)-\frac{f(b^{-}+a_{\ell})p(b^{-}+a_{\ell})}{p(x)}$ at all $x\in\mathcal{S}(p)$. Operator $\mathcal{L}_{p}^{\ell}$ is invertible (with inverse $\mathcal{T}_{p}^{\ell}$) on the subclass of functions in $\mathcal{F}^{(0)}(p)\cap\mathcal{F}^{(1)}(p)$ which, moreover, satisfy $f(b^{-}+a_{\ell})p(b^{-}+a_{\ell})=f(a^{+}-b_{\ell})p(a^{+}-b_{\ell})=0$.

Starting from (2.6) we postulate the next definition.

Definition 2.6** **(Standardizations of the canonical

operator).

Fix $\ell\in\left\{-1,0,1\right\}$ and $\eta\in L^{1}(p)$. The $\eta$-standardized Stein operator is

[TABLE]

acting on the collection $\mathcal{F}(\mathcal{A}_{p}^{\ell,\eta})$ of test functions $g$ such that $\mathcal{L}^{\ell}_{p}\eta(\cdot)g(\cdot-\ell)\in\mathcal{F}_{\ell}^{(1)}(p)$ and $(\mathcal{L}^{\ell}_{p}\eta)\Delta^{-\ell}g\in L^{1}(p)$.

Remark 2.7**.**

The conditions appearing in the definition of $\mathcal{F}(\mathcal{A}_{p}^{\ell,\eta})$ are tailored to ensure that all identities and manipulations follow immediately. For instance, the requirement that $\mathcal{L}^{\ell}_{p}\eta(\cdot)g(\cdot-\ell)\in\mathcal{F}_{\ell}^{(1)}(p)$ in the definition of $\mathcal{F}(\mathcal{A}_{p}^{\ell,\eta})$ guarantees that the resulting functions $\mathcal{A}_{p}^{\ell,\eta}g(x)$ have $p$-mean 0 and the condition $(\mathcal{L}^{\ell}_{p}\eta)\Delta^{-\ell}g\in L^{1}(p)$ guarantees that the expectations of the individual summands on the r.h.s. of (2.8) exist. Again, our assumptions are not transparent; we discuss them in detail in Section 2.3.

The final ingredient for Stein differentiation is the Stein equation:

Definition 2.8** (Stein equation).**

Fix $\ell\in\left\{-1,0,1\right\}$ and $\eta\in L^{1}(p)$. For ${h}\in L^{1}(p)$, the $\mathcal{A}_{p}^{\ell,\eta}$-Stein equation for $h$ is the functional equation $\mathcal{A}_{p}^{\ell,\eta}g(x)=h(x)-\mathbb{E}[h(X)],x\in\mathcal{S}(p)$, i.e.

[TABLE]

A solution to the Stein equation is any function $g\in\mathcal{F}(\mathcal{A}_{p}^{\ell,\eta})$ which satisfies (2.9) for all $x\in\mathcal{S}(p)$.

Our notations lead immediately to the next result.

Lemma 2.9** (Solution to the Stein equation).**

Fix $\eta\in L^{1}(p)$. The Stein equation (2.9) for $h\in L^{1}(p)$ is solved by

[TABLE]

with the convention that $g_{h}^{p,\ell,\eta}(x)=0$ for all $x+\ell$ outside of $\mathcal{S}(p)$.

Proof.

With $g=g_{h}^{p,\ell,\eta}$,

[TABLE]

using Lemma 2.5 for the last step. Hence (2.9) is satisfied for all $x\in\mathcal{S}(p)$. Since, by construction, $g\in\mathcal{F}(\mathcal{A}_{p}^{\ell,\eta})$, the claim follows. ∎

When the context is clear then we drop the superscripts and the subscript in $g$ of (2.10). Before proceeding we provide two examples. The notation $\mathrm{Id}$ refers to the identity function $x\rightarrow\mathrm{Id}(x)=x$.

Example 2.10** (Binomial distribution).**

Let $p(x)=\binom{n}{x}\theta^{x}(1-\theta)^{n-x}$ be the binomial density with parameters $(n,\theta)$ and $\mathcal{S}(p)=[0,n]\cap{\rm I\kern-1.60004ptN}$; assume that $0<\theta<1$. Stein’s method for the binomial distribution was first developed in [32] using $\Delta^{-}$; see also [68, 43].

Picking $\ell=1$, the class $\mathcal{F}_{+}^{(1)}(p)$ consists of functions $f:{\mathbb{Z}}\to{\rm I\kern-1.60004ptR}$ which are bounded on $\mathcal{S}(p)$ and $f(0)=0$. Fixing $\eta(x)=x-n\theta$ gives $\mathcal{L}_{\mathrm{bin}(n,\theta)}^{+}\eta(x)=-(1-\theta)x$ leading to

[TABLE]

with corresponding class $\mathcal{F}\left(\mathcal{A}^{+,\mathrm{Id}}_{\mathrm{bin}(n,\theta)}\right)$ which contains all functions $g:{\mathbb{Z}}\to{\rm I\kern-1.60004ptR}$. The solution to the $\mathcal{A}^{+,\mathrm{Id}}_{\mathrm{bin}(n,\theta)}$-Stein equation (see (2.9)) is

[TABLE]

and $g^{+}(n)=0$.

Picking $\ell=-1$, the class $\mathcal{F}_{-}^{(1)}(p)$ consists of functions $f:{\mathbb{Z}}\to{\rm I\kern-1.60004ptR}$ which are bounded on $\mathcal{S}(p)$ and such that $f(n)=0$. Again fixing $\eta(x)=x-n\theta$ gives $\mathcal{L}_{\mathrm{bin}(n,\theta)}^{-}\eta(x)=-\theta(n-x)$ leading to

[TABLE]

acting on the same class as (2.11). The solution to the $\mathcal{A}^{-,\mathrm{Id}}_{\mathrm{bin}(n,\theta)}$-Stein equation is

[TABLE]

and $g^{-}(0)=0$. The function $-g^{-}$ is studied in [32] where bounds on $\|\Delta^{-}g^{-}\|$ are provided (see equation (10) in that paper); see also Section 2.4 where bounds on $\|g^{-}\|$ are provided.

Example 2.11** (Beta distribution).**

Let $p(x)=x^{\alpha-1}(1-x)^{\beta-1}/B(\alpha,\beta)$ be the beta density with parameters $(\alpha,\beta)$ and $\mathcal{S}(p)=(0,1)$. Stein’s method for the beta distribution was developed in [37, 31] using the Stein operator $\mathcal{A}f(x)=x(1-x)f^{\prime}(x)+(\alpha(1-x)-\beta x)f(x)$. In our notations, we have $\ell=0$ and $\mathcal{F}^{{(1)}}_{0}(p)$ consists of functions $f:{\rm I\kern-1.60004ptR}\to{\rm I\kern-1.60004ptR}$ such that $f(0^{+})p(0^{+})=f(1^{-})p(1^{-})$ and $|(fp)^{\prime}|$ is Lebesgue integrable on $[0,1]$. Fixing $\eta(x)=x-\frac{\alpha}{\alpha+\beta}$ gives $\mathcal{L}_{\mathrm{beta}(\alpha,\beta)}\eta(x)=-\frac{x(1-x)}{\alpha+\beta}$ leading to the operator

[TABLE]

with domain $\mathcal{F}\left(\mathcal{A}^{\mathrm{Id}}_{\mathrm{Beta}(\alpha,\beta)}\right)$ the set of differentiable functions $g:{\rm I\kern-1.60004ptR}\to{\rm I\kern-1.60004ptR}$ such that $x(1-x)g(x)\in\mathcal{F}^{{(1)}}_{0}(p)$ and $x(1-x)g^{\prime}(x)\in L^{1}(p)$. The solution to the $\mathcal{A}^{\mathrm{Id}}_{\mathrm{Beta}(\alpha,\beta)}$-Stein equation is

[TABLE]

The operator $\mathcal{A}^{\mathrm{Id}}_{\mathrm{Beta}(\alpha,\beta)}f$ is, up to multiplication by $\alpha+\beta$, the classical Stein operator $\mathcal{A}f$ for the beta density, see [37, 31] for details and bounds on solutions and their derivatives. See also Section 2.4 where bounds on $\|g\|$ are provided.

In order to propose a more general example, we recall the concept of a Stein kernel, here extended to continuous and discrete distributions alike.

Definition 2.12** (The Stein kernel).**

Let $X\sim p$ have finite mean. The ($\ell$-)Stein kernel of $X$ (or of $p$) is the function

[TABLE]

Metonymously, we refer to the random variable $\tau_{p}^{\ell}(X)$ as the ($\ell$-)Stein kernel of $X.$

Remark 2.13**.**

*The function $\tau_{p}^{\ell}(\cdot)$ is studied in detail for $\ell=0$ in [70, Lecture VI]. This function is particularly useful for Pearson (and discrete Pearson a.k.a. Ord) distributions which are characterized by the fact that their Stein kernel $\tau_{p}^{\ell}$ is a second degree polynomial, see Example 3.8. For more on this topic, we also refer to forthcoming [33] as well as [28, 36, 35] wherein important contributions to the theory of Stein kernels are provided in a multivariate setting. *

The next example gives some ($\ell$-)Stein kernels, exploiting the fact that if the mean of $X$ is $\nu$, then $\mathcal{L}_{p}^{\ell}(\mathrm{Id})(x)=\mathcal{L}_{p}^{\ell}(\mathrm{Id}-\nu)(x).$

Example 2.14**.**

If $X\sim\mathrm{Bin}(n,\theta)$ then using $\eta(x)=x-n\theta$, Example 2.10 gives $\tau_{\mathrm{bin}(n,\theta)}^{+}(x)=(1-\theta)x$ and $\tau_{\mathrm{bin}(n,\theta)}^{-}(x)=\theta(n-x)$. If $X\sim\mathrm{Beta}(\alpha,\beta)$ then Example 2.11 with $\eta(x)=x-\frac{\alpha}{\alpha+\beta}$ gives $\tau_{\mathrm{Beta}(\alpha,\beta)}^{0}(x)=x(1-x)/(\alpha+\beta)$.

Example 2.15** (A general example).**

Let $p$ satisfy Assumption A and suppose that it has finite mean $\nu$. Fixing $\eta(x)=x-\nu$, operator (2.8) becomes

[TABLE]

with corresponding class \mathcal{F}\bigg{(}\mathcal{A}_{p}^{\tau_{p}^{\ell}}\bigg{)} which contains all functions $g:{\rm I\kern-1.60004ptR}\to{\rm I\kern-1.60004ptR}$ such that $\tau_{p}^{\ell}({\cdot})g({\cdot-\ell})\in\mathcal{F}^{(1)}_{\ell}(p)$ and $\tau_{p}^{\ell}\Delta^{-\ell}g\in L^{1}(p)$. Again, we stress that such conditions are clarified in Section 2.3. Using Lemma 2.9, the solution to the $\mathcal{A}_{p}^{\tau_{p}}$ Stein equation is

[TABLE]

Bounds on $\|g\|$ are provided in Section 2.3. Stein’s method based on $\mathcal{A}_{p}^{\tau_{p}^{\ell}}$ is already available in several important subcases, e.g. in [67, 47, 31] for continuous distributions.

The construction is tailored to ensure that all operators have mean 0 over the entire classes of functions on which they are defined. We immediately deduce the following family of Stein integration by parts formulas:

Lemma 2.16** (Stein IBP formula - version 2).**

*Let $X\sim p$. Then *

[TABLE]

for all $f\in L^{1}(p)$ and all $g\in\mathrm{dom}(\Delta^{-\ell})$ such that $\mathcal{L}_{p}^{\ell}f(\cdot)g(\cdot-\ell)\in\mathcal{F}_{\ell}^{(1)}(p)$ and ${\mathcal{L}_{p}^{\ell}f(\cdot)\Delta^{-\ell}g(\cdot)}\in L^{1}(p)$.

Proof.

Identity (2.12) follows directly from the Stein product rule in [53, Theorem 3.24] or by using the fact that expectations of the operators in (2.8) are equal to 0. ∎

We stress the fact that in the formulation of Lemma 2.16 the test functions $f$ and $g$ do not play a symmetric role. If $g\in L^{1}(p)$ then the right hand side of (2.12) is the covariance $\mathrm{Cov}(f(X),g(X))$. We shall use this heavily in our future developments. Similarly as for Lemma 2.3, the conditions under which Lemma 2.16 applies are not transparent in their present form. In Section 2.3 various explicit sets of conditions are provided under which the IBP (2.12) is applicable.

2.2 Representations of the inverse Stein operator

This section contains the first main results of the paper, namely probabilistic representations for this operator. Such representations are extremely useful for manipulations of the operators. We start with a simple rewriting of $\mathcal{L}_{p}^{\ell}$. Given $\ell\in\left\{-1,0,1\right\}$, recall the notation $a_{\ell}=\mathbb{I}[\ell=1]$ and define

[TABLE]

Such generalized indicator functions particularize, in the three cases that interest us, to $\chi^{0}(x,y)=\mathbb{I}[x\leq y]$ \big{(}\ell=0\big{)}, $\chi^{-}(x,y)=\mathbb{I}[{x\leq y}]$ \big{(}\ell=-1\big{)} and $\chi^{+}(x,y)=\mathbb{I}[{x<y}]$ \big{(}\ell=1\big{)}. Their properties lead to some form of “calculus” which shall be useful in the sequel.

Lemma 2.17** (Chi calculation rules).**

The function $\chi^{\ell}(x,y)$ is non-increasing in $x$ and non-decreasing in $y$. For all $x,y$ we have

[TABLE]

Moreover,

[TABLE]

Let $p$ with support $\mathcal{S}(p)$ satisfy Assumption A. Then for any $f\in L^{1}(p)$ it is easy to check from the definition (2.7) that

[TABLE]

Next, define

[TABLE]

for all $x\in\mathcal{S}(p)$ and 0 elsewhere. This function is used in the following representation formula for the Stein inverse operator:

Lemma 2.18** (Representation formula I).**

Let $X,X_{1},X_{2}$ be independent copies of $X\sim p$ with support $\mathcal{S}(p)$. Then, for all $f\in L^{1}(p)$ we have

[TABLE]

Proof.

The $L^{1}(p)$ condition on $f$ suffices for the expectation on the r.h.s. of (2.18) to be finite for all $x\in\mathcal{S}(p)$. Suppose without loss of generality that $\mathbb{E}[f(X)]=0$. Using that $X_{1},X_{2}$ are i.i.d., we reap

[TABLE]

where in the third line we used the fact that $\mathbb{E}[f(X)\mathbb{I}[\ell=0]\mathbb{I}[X=x]]=0$ under the stated assumptions. For the same reasons, we have $\mathbb{E}[\chi^{\ell}(X,x)+\chi^{-\ell}(x,X)]=1$ for all $x\in\mathcal{X}$ and all $\ell\in\left\{-1,0,1\right\}$. The conclusion follows by recalling (2.16). ∎

The function defined in (2.17) allows to perform “probabilistic integration” as follows: if $f\in\mathrm{dom}(\Delta^{-\ell})$ is such that $(\Delta^{-\ell}f)$ is integrable on $[x_{1},x_{2}]\cap\mathcal{S}(p)$ then

[TABLE]

for all $x_{1}<x_{2}\in\mathcal{S}(p)$. If, furthermore, $f\in L^{1}(p)$ then (by a conditioning argument)

[TABLE]

Equation (2.19) leads to the next representation formula for the inverse Stein operator.

Lemma 2.19** (Representation formula II).**

Let $X\sim p$. Define the kernel $K_{p}$ on $\mathcal{S}(p)\times\mathcal{S}(p)$ by

[TABLE]

*Then $K_{p}^{\ell}(x,x^{\prime})$ is symmetric and positive. Moreover, for all $f\in\mathrm{dom}(\Delta^{-\ell})$ such that $f\in L^{1}(p)$ we have, *

[TABLE]

Proof.

Symmetry of $K_{p}^{\ell}$ is immediate. To see that it is positive, applying first (2.15) and then (2.14),

[TABLE]

which is necessarily positive. To prove (2.21), we insert (2.19) into (2.18), to obtain

[TABLE]

For all $x,x^{\prime}\in\mathcal{S}(p)$, by (2.15),

[TABLE]

Using (2.22), we recognize the kernel $K_{p}^{\ell}(x,x^{\prime})$ in the numerator, and identity (2.21) follows. ∎

Example 2.20**.**

Representations (2.18) and (2.21) can easily be applied to obtain representations for the Stein kernel $\tau_{p}^{\ell}(x)$:

[TABLE]

In particular the Stein kernel is positive on $\mathcal{S}(p)$.

Identity (2.18) seems to be new, although it is present in non-explicit form in [22, Equation (4.16)]. Representation (2.21) is, in the continuous $\ell=0$ case, already available in [64]. The kernel $K_{p}^{\ell}(x,x^{\prime})$ is a classical object in the theory of covariance representations and inequalities; an early appearance is attributed by [59] to [41] (see [42, pp 57–109] for an English translation). The perhaps not very surprising extension to the discrete case is, to the best of our knowledge, new.

As a first result from our set-up, (2.21) applied to the function $f(x)=\mathcal{T}_{p}^{\ell}1(x)$ immediately gives the following:

Proposition 2.21** (Menz-Otto formula).**

Suppose that the constant function $1$ belongs to $\mathcal{F}_{\ell}^{(1)}(p)$, that $-\Delta^{-\ell}\mathcal{T}_{p}^{\ell}1(x)>0$ for almost all $x\in\mathcal{S}(p)$ and $\Delta^{-\ell}(\mathcal{T}_{p}^{\ell}1)\in L^{1}(\mu)$. Then, for every $x^{\prime}\in\mathcal{S}(p)$, the function

[TABLE]

is a density on $\mathcal{S}(p)$ with respect to $\mu$.

Proof.

From (2.21) with $f(x)=\mathcal{T}_{p}^{\ell}1(x)=\frac{\Delta^{\ell}(p(x))}{p(x)}$ and $\mathcal{L}_{p}^{\ell}f(x)=1$,

[TABLE]

Since $0\leq K_{p}^{\ell}(x,x^{\prime})\leq 1$, the integral exists because $-\Delta^{-\ell}\mathcal{T}_{p}^{\ell}1(x)\frac{K_{p}^{\ell}(x,x)}{p(x)}\in L^{1}(p)$ and, by assumption, $-\Delta^{-\ell}\mathcal{T}_{p}^{\ell}1(x)>0$. Hence the assertion follows. ∎

Remark 2.22**.**

If $\frac{K_{p}^{\ell}(x,x)}{p(x)}$ is bounded, then the assumptions in Proposition 2.21 are satisfied as soon as $-\Delta^{-\ell}\mathcal{T}_{p}^{\ell}1\in L^{1}({\mu})$. The proposition thus applies when $\ell=0$ and $p(x)=e^{-H(x)}$ with $H$ a strictly convex function such that $\lim_{x\to\pm\infty}H^{\prime}(x)=0$. This puts us in the context studied by [55] and formula (2.23) is equivalent to their [55, Equation (14)]; we return to this in Section 3.

The next proposition gives some properties of $K_{p}^{\ell}(x,x^{\prime})$.

Proposition 2.23**.**

(i) It holds that for all $x,x^{\prime}\in\mathcal{S}(p),$ that $K_{p}^{\ell}(x,x^{\prime})\leq{K_{p}^{\ell}(\min(x,x^{\prime}),\min(x,x^{\prime}))}$. (ii) If $\mathbb{E}[\chi^{\ell}(X,x)]/p(x)$ is non decreasing, then the function $x\mapsto{K_{p}^{\ell}(x,x^{\prime})}/{p(x)}$ is non-decreasing for $x<x^{\prime}$. (iii) If $\mathbb{E}[\chi^{-\ell}(x,X)]/p(x)$ is non increasing, then the function $x\mapsto{K_{p}^{\ell}(x,x^{\prime})}/{p(x)}$ is non-increasing for $x>x^{\prime}$.

Proof.

To see (i), we start from (2.22), $K_{p}^{\ell}(x,x^{\prime}){=\mathbb{E}\left[\chi^{\ell}(X,\min(x,x^{\prime}))\right]\mathbb{E}\left[\chi^{-\ell}(\max(x,x^{\prime}),X)\right]}$ and by Lemma 2.17, $\chi^{-\ell}(u,x)$ is non-increasing in $u$: $\chi^{-\ell}(\max(x,x^{\prime}),X)\leq\chi^{-\ell}(\min(x,x^{\prime}),X)$. We deduce that

[TABLE]

Assertion (i) follows by reverting the argument, because $(\chi^{\ell}(X,\min(x,x^{\prime}))^{2}=\chi^{\ell}(X,\min(x,x^{\prime}))$. To see (ii), assume that $\mathbb{E}[\chi^{\ell}(X,x)]/p(x)$ is non-decreasing. Then with (2.22), for $x<x^{\prime}$,

[TABLE]

the second factor is a constant, and the first factor is assumed to be non-decreasing. Hence the assertion follows. For (iii), assume that $\mathbb{E}[\chi^{-\ell}(x,X)]/p(x)$ is non-increasing; then similarly as above, for $x>x^{\prime}$,

[TABLE]

the first factor is constant, and the second factor is non increasing. Hence the assertion follows. ∎

Figures 1 and 2 display the functions $x\mapsto{K_{p}^{\ell}(x,x^{\prime})}/{p(x)}$ (for various values of $x^{\prime}$) and $x\mapsto{K_{p}^{\ell}(x,x)}/{p(x)}$ for the standard normal and several choices of the parameters in beta, gamma, binomial, Poisson and hypergeometric distributions.

Example 2.24**.**

The following facts are easy to prove:

If $p(x)$ is the standard normal distribution then ${K_{p}^{0}(x,x)}/{p(x)}$ behaves as $1/|x|$ for large $|x|$, see Figure 2(a). 2. 2.

If $p(x)$ is gamma then ${K_{p}^{0}(x,x)}/{p(x)}$ behaves as a constant for large $x$, see Figure 2(c). 3. 3.

*The function $x\mapsto{K_{p}^{0}(x,x)}/{p(x)}$ is not in $L^{1}(p)$ for $p$ a Cauchy distribution. * 4. 4.

If $p$ is strictly-log concave then ${K_{p}^{\ell}(x,x)}/{p(x)}$ is bounded.

2.3 Sufficient conditions and integrability

As anticipated, we now study the conditions under which the IBP Lemmas 2.3 and 2.16 hold. All proofs are technical manipulations of basic calculus and relegated to the Appendix A.

We start by the decryption of the conditions for Lemma 2.3. Recall the notations $a_{\ell}$ and $b_{\ell}$ from (2.3). Furthermore if $\ell=0$ we write $f(a^{+})=\lim_{x\to a,x>a}f(x)$ and $f(b^{-})=\lim_{x\to b,x<b}f(x)$. In the case that $a=-\infty$ or $b=\infty$, for $\ell\in\{-1,0,1\}$, we write $f(-\infty^{+})=\lim_{x\to-\infty}f(x)$ and $f(\infty^{-})=\lim_{x\to\infty}f(x)$. To simplify notation, if $\ell\in\{-1,1\}$ and $a\neq-\infty$, we write $f(a^{+})=f(a)$, and similarly, if $b\neq\infty,f(b^{-})=f(b)$.

Proposition 2.25** **(Sufficient conditions for IBP – version

1).

Let $f\in\mathrm{dom}(p,\Delta^{\ell})$ and $g\in\mathrm{dom}(\Delta^{-\ell})$. In order for (2.5) to hold it suffices that they jointly satisfy the following conditions

[TABLE]

For ease of future reference, we spell out (2.25) in the three cases that interest us:

[TABLE]

We now derive a set of (almost) necessary and sufficient conditions under which (2.12) holds.

Proposition 2.26** (Sufficient conditions for IBP – version 2).**

Let $g\in\mathrm{dom}(\Delta^{-\ell})$. In order for (2.12) to hold, it is necessary and sufficient that they jointly satisfy the three following conditions:

[TABLE]

Requirement (2.26) is natural and condition (2.28) is mild as it is satisfied as soon as $g$ and/or $f$ are well behaved at the edges of the support. Condition (2.27) (which is already stated in the original statement of Lemma 2.16) is harder to fathom. In order to make it even more readable, and facilitate the connexion with the literature, we specialise the conditions further in our next result.

Proposition 2.27**.**

Let $f,g$ and $fg\in L^{1}(p)$. If $g\in\mathrm{dom}(\Delta^{-\ell})$ is of bounded variation and satisfies the following two conditions:

$g(a^{+}-b_{\ell}-\ell)\mathbb{P}(X\leq a^{+}-a_{\ell}-b_{\ell})=0$* and $g(b^{-}+a_{\ell}-\ell)\mathbb{P}(X\geq b^{-}+a_{\ell}+b_{\ell})=0$* 2. 2.

$g(a^{+}-b_{\ell}-\ell)\mathbb{E}[|f(X)|\chi^{\ell}(X\leq a^{+}-b_{\ell})]=0$* and $g(b^{-}+a_{\ell}-\ell)\mathbb{E}[|f(X)|\chi^{-\ell}(b^{-}+a_{\ell},X)]=0$,*

then (2.28) holds. In particular if $f$ is bounded or in $L^{2}(p)$, then the condition 2 above is implied by condition 1.

Remark 2.28**.**

This assumption is closer to what is to be found in the literature, see e.g. [64] in the case $\ell=0$. The main difference between the classical assumptions and ours is that we only impose conditions on one of the functions. We stress that there is a certain degree of redundancy in the items 1 and 2 together with the assumption that $g\in L^{1}(p)$ and is of bounded variation; the statement could be shortened at the loss of readability.

In the sequel, to preserve as much generality as possible and not overburden the statements, we will simply require that “the assumptions of Lemma 2.16 are satisfied.”

2.4 The inverse Stein operator

We conclude this section by exploring easy consequences of the representations from Section 2.2. These results are also of independent interest to practitioners of Stein’s method.

Lemma 2.29**.**

If $f,\mathcal{L}_{p}^{\ell}f(X)\in L^{1}(p)$ then

[TABLE]

where $(\cdot)^{+}$ denotes the positive part of $(\cdot)$. In particular, if the conditions of Lemma 2.16 are satisfied with $f(x)=g(x)=\mathrm{Id}$, then $\mathbb{E}[\tau_{p}^{\ell}(X)]=\mathrm{Var}(X).$

Proof.

Representation (2.18) gives \mathbb{E}\big{[}-\mathcal{L}_{p}^{\ell}f(X)\big{]}=\mathbb{E}[(f(X_{2})-f(X_{1}))\Phi_{p}^{\ell}(X_{1},X,X_{2})]. Using (2.19) with $f(x)=x$, we have $\mathbb{E}[\Phi_{p}^{\ell}(x_{1},X,x_{2})]=(x_{2}-x_{1})^{+}$. Hence, after conditioning with respect to $X_{1},X_{2}$, the first equality in (2.29) follows. The second equality follows by symmetry. The second claim is immediate under the stated assumptions. ∎

Remark 2.30**.**

Once again, our assumptions are minimal but not transparent. It is easy to spell out these conditions explicitly for any specific target. For instance if $X$ has bounded support or support ${\rm I\kern-1.60004ptR}$ then finite variance suffices.

Proposition 2.31**.**

Suppose that all test functions satisfy the conditions in Lemma 2.16. Let $\|f\|_{{\mathcal{S}}(p),\infty}=\sup_{x\in{\mathcal{S}}(p)}|f(x)|.$

If $f$ is monotone then $\mathcal{L}_{p}^{\ell}f(x)$ does not change sign. 2. 2.

(Uniform bounds Stein bounds) Consider, for $h$ and $\eta$ in $L^{1}(p)$ the function

[TABLE]

defined in (2.10) which solves the $\eta$-Stein equation (2.9) for $h$. If $\eta$ is monotone and $|h(x)-h(y)|\leq k|\eta(x)-\eta(y)|$ for all $x,y\in\mathcal{S}(p)$, then

[TABLE]

In particular, if $h\in L^{1}(p)$ is Lipschitz continuous with Lipschitz constant $1$ then the above applies with $\eta(x)=x$, and $\|g_{h}^{p,0,\mathrm{Id}}\|_{{\mathcal{S}}(p),\infty}\leq 1$. 3. 3.

(Non uniform bounds Stein bounds)

[TABLE]

for all $x\in\mathcal{S}(p)$.

Proof.

Recall representation (2.18) which states that

[TABLE]

If $f(x)$ is monotone then $f(X_{2})-f(X_{1})$ is of constant sign conditionally on $\chi^{\ell}(X_{1},x)\chi^{-\ell}(x,X_{2})={\mathbb{I}[X_{1}+a_{\ell}\leq x\leq X_{2}-b_{\ell}]=1}$, because on this event, $X_{1}\leq X_{2}-\mathbb{I}[\ell\neq 0]$. Hence the first assertion follows. 2. 2.

Suppose that the function $\eta$ is strictly decreasing. By definition of $g$ we have, under the stated conditions,

[TABLE]

for $x\in\mathcal{S}(p)$. 3. 3.

By (2.18),

[TABLE]

which leads to the conclusion.

∎

Example 2.32**.**

If $p=\phi$ is the standard Gaussian with cdf $\Phi$, then $\ell=0$ and the third bound in Proposition 2.31 reduces to ${2}||f||_{\infty}\frac{\Phi(x)(1-\Phi(x))}{\phi(x)}$. The ratio {\Phi(x)\big{(}1-\Phi(x)\big{)}}/{\phi(x)} is closely related to Mill’s ratio of the standard normal law. The study of such a function is classical and much is known. For instance, we can apply [9, Theorem 2.3] to get

[TABLE]

for all $x\geq 0$. Moreover, {\Phi(x)\big{(}1-\Phi(x)\big{)}}/{\phi(x)}\leq{\Phi(0)\big{(}1-\Phi(0)\big{)}}/{\phi(0)}=1/2\sqrt{\pi/2}\approx 0.626. In particular Proposition 2.31 recovers the well-known bound $\|\mathcal{L}_{p}^{\ell}f\|_{\infty}\leq\sqrt{\pi/2}\|f\|_{\infty}$, see e.g. [57, Theorem 3.3.1].

3 Covariance identities and inequalities

We start with an easy lower bound inequality, which follows immediately from Lemma 2.3.

Proposition 3.1** (Cramer-Rao type bound).**

Let $g\in L^{2}(p)$. For any $f\in\mathcal{F}_{\ell}^{(1)}(p)$ such that $\mathcal{T}_{p}^{\ell}f\in L^{2}(p)$ and the assumptions of Lemma 2.3 are satisfied:

[TABLE]

with equality if and only if there exist $\alpha,\beta$ real numbers such that $g(x)=\alpha\mathcal{T}_{p}^{\ell}f(x)+\beta$ for all $x\in\mathcal{S}(p)$.

Proof.

The lower bound (3.1) follows from the fact that $\mathcal{T}_{p}^{\ell}f(X)\in\mathcal{F}^{(0)}(p)$ for all $f\in\mathcal{F}^{{(1)}}_{\ell}(p)$ by Lemma 2.2. Therefore, from (2.5), we have

[TABLE]

by the Cauchy-Schwarz inequality. ∎

Upper bounds require some more work. We start with an easy consequence of our framework.

Corollary 3.2** **(First order covariance

identities).

*For all $f,g$ that jointly satisfy the assumptions of Lemma 2.16, we have *

[TABLE]

Moreover, if choice $f=\mathrm{Id}$ is allowed, then

[TABLE]

Remark 3.3**.**

Identity (3.2) is provided in [55] (see their equation (11)) in the case $\ell=0$ for a log-concave density. Some of the history of this identity, including the connection with a classical identity of Hoeffding [41], is provided in [66, Section 2]. The earliest version of the same identity (still for $\ell=0$) we have found in [29], along with applications to measures of correlation as well as further references. A similar identity is provided in [55], without explicit conditions; a clear statement is given in [66, Corollary 2.2] where the identity is proved for absolutely continuous $f\in L^{r}$ and $g\in L^{s}$ with conjugate exponents. Our approach shows that it suffices to impose regularity on one of the functions for the identity to hold.

Proof.

Let $\bar{f}(x)=f(x)-\mathbb{E}[f(X)]$. Note that $\Delta^{\ell}{\bar{f}}=\Delta^{\ell}f.$ To obtain (3.2) we start from (2.12) and note that if $f,g$ satisfy the assumptions of Lemma 2.16, then

[TABLE]

From this equation, (3.3) follows immediately. Applying (2.21) we obtain

[TABLE]

which gives the claim after removing the conditioning. ∎

Example 3.4**.**

Example 2.14 and identity (3.3) give the following covariance identities.

• •

Binomial distribution: For all functions $g:{\mathbb{Z}}\to{\rm I\kern-1.60004ptR}$ that are bounded on $[0,n]$,

[TABLE]

Combining the two identities we also arrive at

[TABLE]

with $\nabla_{\mathrm{bin}(n,{\theta})}$ the “natural” binomial gradient $\nabla_{\mathrm{bin}(n,{\theta})}g(x)=\frac{x}{n}\Delta^{-}g(x)+\frac{n-x}{n}\Delta^{+}g(x)$ from **[40]**.

• •

Beta distribution: For all absolutely continuous $g$ such that $\mathbb{E}\left[|X(1-X)g^{\prime}(X)|\right]<\infty$,

[TABLE]

It is of interest to work as in [45] to obtain a corresponding upper bound, which would provide some “weighted Poincaré inequality” such as those described in [64]. The representation formulae (3.2) turns out to simplify the work considerably.

Theorem 3.5**.**

Fix $h\in L^{1}(p)$ a decreasing function. For all $f,g$ which $\mathcal{T}_{p}^{\ell}c\in L^{2}(p)$satisfy the assumptions of Lemma 2.16 we have

[TABLE]

with equality if and only if there exist $\alpha_{i},i=1,\ldots,4$ real numbers such that $f(x)=\alpha_{1}h(x)+\alpha_{2}$ and $g(x)=\alpha_{3}h(x)+\alpha_{4}$ for all $x\in\mathcal{S}(p)$.

Proof.

We simply apply (3.2) and the Cauchy-Schwarz inequality to obtain

[TABLE]

using (2.21) leads to the inequality.

The only part of the claim that remains to be proved concerns the saturation condition in the inequality. This follows from the Cauchy-Schwarz inequality which is an equality if and only if $\Delta^{-\ell}f(x)/\Delta^{-\ell}h(x)\propto\Delta^{-\ell}g(x^{\prime})/\Delta^{-\ell}h(x^{\prime})$ is constant throughout $\mathcal{S}(p)$. This is only possible under the stated condition. ∎

Remark 3.6**.**

Theorem 3.5 can be refined using the exact expression for the remainder in the Cauchy-Schwarz inequality, given by the Lagrange-type identity

[TABLE]

with $X_{1},X_{2},X_{3},X_{4}$ independent copies with density $p$ and $f,g\in L^{2}(p)$. Fix $h\in L^{1}(p)$ a decreasing function such that $\|h\|_{{\mathcal{S}}(p),\infty}<\infty$. For all $f,g$ which satisfy the assumptions of Lemma 2.16 we have

[TABLE]

with

[TABLE]

In particular when $h(x)=x$ the remainder term simplifies to

[TABLE]

Combining Proposition 3.1 and Theorem 3.5 (applied with $f=g$) we arrive at the following result (applied to a smaller class of functions $h$) which, as we shall argue below, share a similar flavour to the upper and lower bounds from Theorem 1.1.

Corollary 3.7** (Klaassen bounds, revisited).**

For any decreasing function $h\in L^{2}(p)$ and all $g$ such that Lemma 2.16 applies (with $f=g$), we have

[TABLE]

Equality in the upper bound holds if and only if there exists constants $\alpha,\beta$ such that $g(x)=\alpha h(x)+\beta$.

Proof.

For the lower bound, we apply Proposition 3.1 with $f(x)=-\mathcal{L}_{p}^{\ell}h(x)$ so that $\mathcal{T}_{p}^{\ell}c(x)=h(x)--\mathbb{E}[h(X)].$ For the upper bound we use Theorem 3.5 with $f=g$. ∎

Example 3.8** (Pearson and Ord families).**

Tables 1, 2, and 3 present the results for random variables whose distribution belongs to the Pearson and Ord families of distributions. A random variable $X\sim p$ belongs to the integrated Pearson family if $X$ is absolutely continuous and there exist $\delta,\beta,\gamma\in{\rm I\kern-1.60004ptR}$ not all equal to 0 such that \tau_{p}^{\ell}(x)\big{(}:=-\mathcal{L}_{p}^{\ell}(\mathrm{Id})\big{)}=\delta x^{2}+\beta x+\gamma for all $x\in\mathcal{S}(p)$. Similarly, $X\sim p$ belongs to the cumulative Ord family if $X$ is discrete and there exist $\delta,\beta,\gamma\in{\rm I\kern-1.60004ptR}$ not all equal to 0 such that \tau_{p}^{\ell}(x)\big{(}:=-\mathcal{L}_{p}^{\ell}(\mathrm{Id})\big{)}=\delta x^{2}+\beta x+\gamma for all $x\in\mathcal{S}(p)$. The bounds for these distributions generalize the results e.g. from [2].

Remark 3.9** (About the connection with Klaassen’s bounds).**

The bounds in Corollary 3.7 and those from Theorem 1.1 are obviously of a similar flavour. Upon closer inspection, however, the connection is not transparent. In order to clarify this point, we follow [45] and restrict our attention to kernels of the form

[TABLE]

for some $\zeta\in{\rm I\kern-1.60004ptR}$. In our notations, these become

[TABLE]

for $\ell\in\left\{-1,0,1\right\}.$

We first tackle the relation between the main arguments of the bounds, namely $G(x)$ and $g(x)$. Given a measurable function $g$, we mimic the statement of Theorem 1.1 and introduce the generalized primitive $G(x)=G_{\zeta}^{\ell}(x):=\int\rho_{\zeta}^{\ell}(x,y)g(y)\mu(\mathrm{d}y)+c$ with $c$ arbitrary, fixed w.l.o.g. to 0. Again in our notations, this becomes

[TABLE]

By construction, $\Delta^{-\ell}G_{\zeta}^{\ell}(x)=g(x)$ for all $\zeta$ and all $\ell$, as expected. Nevertheless, in order for $G_{\zeta}^{\ell}(x)$ to be well-defined, strong (joint) assumptions on $g$ and $\zeta$ are required; for instance, if $g(x)=1$ then $\zeta$ must be finite and $G_{\zeta,c}^{\ell}(x)=x-\zeta$ while if $g(x)$ has $p$-mean 0 then the values $\zeta=\pm\infty$ are allowed.

Next, we examine the connection between the lower bound (1.7) and the lower bound of (3.6). Let $k\in L^{2}(p)$ have $p$-mean 0. Then

[TABLE]

so that

[TABLE]

and thus (1.7) follows from the lower bound of (3.6).

Finally, we consider the upper bounds (1.6) and (3.6). Let $H(x)=H_{\zeta}^{\ell}(x)$ be a generalized primitive of some nonnegative function $h$. The same manipulations as above lead to

[TABLE]

If, following [45], we choose $\zeta$ in such a way that $\mathbb{E}[H(X)]=0$ (this is equivalent to requiring $\int_{\zeta+a_{\ell}}^{b}h(y)(1-P(y+b_{\ell}))\mu(\mathrm{d}y)=\int_{a}^{\zeta-b_{\ell}}h(y)(P(y-a_{\ell}))\mu(\mathrm{d}y)$) then we see that the upper bound in (3.6) is equivalent to (1.6).

Of course there is some gain in generality at allowing for a general kernel $\rho$ as in Theorem 1.1, though this comes at the expense of readability: given a positive function $h$, understanding the form of function $H$ is actually non trivial and our result illuminates Klaassen’s discovery by providing the connection with Stein characterizations.

4 About the weights

The freedom of choice in the test functions $h$ appearing in the bounds invite a study of the impact of the choice of $h$ on the validity and quality of the resulting inequalities.

4.1 Score function and the Brascamp-Lieb inequality

The form of the lower bound in Proposition 3.1 encourages the choice $f(x)=1$. This is only permitted if the constant function $1\in\mathcal{F}_{\ell}^{(1)}(p)$ and $\mathbb{E}\left[\left(\mathcal{T}_{p}^{\ell}1(X)\right)^{2}\right]<\infty$; these are two strong assumptions which exclude some natural targets such as e.g. the exponential or beta distributions. If this choice is permitted, then we reap the lower bound

[TABLE]

with $I^{\ell}(p)=\left[\left(\mathcal{T}_{p}^{\ell}1(X)\right)^{2}\right]$.

The function $\mathcal{T}_{p}^{\ell}1(x)=\frac{\Delta^{\ell}(p(x))}{p(x)}$ is some form of generalized score function and $I^{\ell}(p)$ a generalized Fisher information. Indeed, if $\ell=0$ and $X\sim p$ is absolutely continuous, then $\mathcal{T}_{p}^{\ell}1(x)=(\log p(x))^{\prime}$ is exactly the (location) score function of $p$ and $I^{(0)}(p)$ is none other than the (location) Fisher information of $p$. More generally we note that if $1\in\mathcal{F}_{\ell}^{(1)}(p)$ then $\mathbb{E}[\mathcal{T}_{p}^{\ell}1(X)]=0$ and, by Lemma 2.3, it satisfies

[TABLE]

for all appropriate $g$; this further reinforces the analogy.

The corresponding upper bound from (3.5) is obtained for $h(x)=\mathcal{T}_{p}^{\ell}1(x)$ in (3.6). Suppose that $p(b^{-}+a_{\ell})=p(a^{+}-b_{\ell})=0$. By construction, $\mathcal{L}_{p}^{\ell}h(x)=\mathbb{I}_{\mathcal{S}(p)}(x)$. If we can further suppose that $\mathcal{T}_{p}^{\ell}1(x)$ is a decreasing function then

[TABLE]

Taking $g=f$ we deduce the following result whose continuous version (i.e. the case $\ell=0$) dates back to [11].

Corollary 4.1** (Brascamp-Lieb inequality).**

Under the same conditions as Proposition 2.21 we have

[TABLE]

for all $g$ such that $\mathcal{T}_{p}^{\ell}1,g$ satisfy together the assumptions of Lemma 2.16.

We conclude with a generalized version of the elegant inequality due to [55, Lemma 2.11], in the form stated in [19, Equation (1.5)].

Corollary 4.2** (Asymmetric Brascamp-Lieb inequality).**

Under the same conditions as above, if ${-}\Delta^{-\ell}\mathcal{T}_{p}^{\ell}1\in L^{1}(\mu)$ then

[TABLE]

for all $f,g$ in $L^{2}(p)$.

Proof.

Under the stated assumptions, we may apply (3.2) to get, after some notational reshuffling,

[TABLE]

where the last line follows by conditioning on $X^{\prime}$ and applying Proposition 2.21. ∎

4.2 Stein kernel and Cacoullos’ bound

It is natural to consider test function $h=-\mathrm{Id}$ in Theorem 3.5. Since $\Delta^{-\ell}h(x)=1$, we obtain

[TABLE]

In particular if $g=f$ then

[TABLE]

in which one recognizes the upper bounds from [12] and also, when $\ell=0$, [64]. The corresponding lower bound in (3.1) is obtained for $f(x)=\tau_{p}^{\ell}(x)$ for which $\mathcal{T}_{p}^{\ell}f(x)=x\mathbb{I}_{\mathcal{S}(p)}(x)$, and the overall bound becomes

[TABLE]

Example 4.3**.**

In our examples (4.3) gives the following covariance identities.

• •

Binomial distribution: Let $X\sim\mathrm{Bin}(n,\theta)$ as in Example 2.10. From Example 2.14 we obtain the upper and lower bounds

[TABLE]

• •

Beta distribution: From Example • ‣ 3.4, for the $\mathrm{Beta}(\alpha,\beta)$-distribution with variance $\frac{\alpha\beta}{(\alpha+\beta)^{2}(\alpha+\beta+1)}$,

[TABLE]

The particular case of other Pearson/Ord distributions is detailed in Tables 1, 2 and 3. The tables include the Binomial distribution and the Beta distribution for easy reference. The Stein operators which are given are those from Example 2.15.

4.3 Eigenfunctions of the adjoint Stein operator

A final interesting choice is $h$ in Theorem 3.5 such that the corresponding weight $\frac{-\mathcal{L}_{p}^{\ell}h(x)}{\Delta^{-\ell}h(x)}$ is constant, i.e. any function $h$ such that there exists $\lambda\in{\rm I\kern-1.60004ptR}$ for which

[TABLE]

By construction, such functions are solution to the eigenfunction problem

[TABLE]

where operator $\mathcal{R}_{p}^{\ell}h:=\mathcal{T}_{p}^{\ell}(\Delta^{-\ell}h)$ is self-adjoint in the sense of that

[TABLE]

for all $f,g$ such that Lemmas 2.3 and 2.16 apply.

Acknowledgements

The research of YS was partially supported by the Fonds de la Recherche Scientifique – FNRS under Grant no F.4539.16. ME acknowledges partial funding via a Welcome Grant of the Université de Liège and via the Interuniversity Attraction Pole StUDyS (IAP/P7/06). YS also thanks Lihu Xu for organizing the “Workshop on Stein’s method and related topics” at University of Macau in December 2018, and where a preliminary version of this contribution was first presented. GR and YS thank Emilie Clette for fruitful discussions on a preliminary version of this work. YS thanks Céline Esser for many fruitful discussions. We also thank Benjamin Arras for several pointers to relevant literature, as well as corrections on the first draft of the paper.

[FIGURE:]

[FIGURE:]

[FIGURE:]

Appendix A Proofs from Section 2.3

Proof of Proposition 2.25.

In order for (2.5) to hold we need (i) $f(\cdot)g(\cdot-\ell)\in\mathcal{F}^{(1)}_{\ell}(p)$ and (ii) $f(\cdot)\Delta^{-\ell}g(\cdot)\in L^{1}(p)$. Condition (ii) is satisfied under (2.24). By definition of $\mathcal{F}^{(1)}_{\ell}(p)$, condition (i) holds if the following three conditions apply: (iA) $f(\cdot)g(\cdot-\ell)\in\mathrm{dom}(p,\Delta^{\ell})$, (iB) \Delta^{\ell}\big{(}p(\cdot)f(\cdot)g(\cdot-\ell)\big{)}\mathbb{I}[\mathcal{S}(p)]\in L^{1}(\mu) and (iC) $\mathbb{E}\left[\mathcal{T}_{p}^{\ell}f(X)g(X-\ell)\right]=0$. The proof hinges on product formula (2.4) which yields:

[TABLE]

for all $x\in\mathcal{S}(p)$. In light of this, condition (iA) is implied by the requirement that $f\in\mathrm{dom}(p,\Delta^{\ell})$ and $g\in\mathrm{dom}(\Delta^{-\ell})$. Similarly, because condition (iB) is equivalent to \frac{\Delta^{\ell}\big{(}p(\cdot)f(\cdot)g(\cdot-\ell)\big{)}}{p(\cdot)}\in L^{1}(p), we see that it is guaranteed by (2.24). Finally, applying (2.2), we that (iC) follows from (2.25). Hence Condition (i) holds under the stated assumptions.

∎

Proof of Proposition 2.26.

In order for (2.12) to hold, it is necessary and sufficient that (i) $f\in L^{1}(p)$, $g\in\mathrm{dom}(\Delta^{-\ell})$, (ii) \big{(}\mathcal{L}_{p}^{\ell}f(\cdot)\big{)}g(\cdot-\ell) and (iii) $\mathcal{L}_{p}^{\ell}f(\Delta^{-\ell}g)\in L^{1}(p)$. Conditions (i) and (iii) are stated explicitly and all that remains is to check that (ii) is equivalent to the stated assumptions. As before, we recall that (ii) is equivalent to (iiA) \big{(}\mathcal{L}_{p}^{\ell}f(\cdot)\big{)}g(\cdot-\ell)\in\mathrm{dom}(p,\Delta^{\ell}); (iiB) \Delta^{\ell}\bigg{(}p(\cdot)\big{(}\mathcal{L}_{p}^{\ell}f(\cdot)\big{)}g(\cdot-\ell)\bigg{)}/p(\cdot)\in L^{1}(p); (iiC) \mathbb{E}\left[\mathcal{T}_{p}^{\ell}\left(\big{(}\mathcal{L}_{p}^{\ell}f(\cdot)\big{)}g(\cdot-\ell)\right)(X)\right]=0. As in the proof of Proposition 2.25, the result hinges on the product rule (2.4) which now reads

[TABLE]

Hence condition (iiA) holds solely under the assumption that $g\in\mathrm{dom}(\Delta^{-\ell})$, condition (iiA) holds under (2.26) and (2.27). Finally, (2.28) guarantees that (iiC) is satisfied. ∎

Proof of Proposition 2.27.

We want to apply Proposition 2.26; hence we check each condition in Proposition 2.26 separately. By assumption, (2.26) is satisfied and $g\in\mathrm{dom}(\Delta^{-\ell})$.

• •

For Assumption (2.27): First suppose that $g$ is monotone increasing. It is to show that $\mathcal{L}_{p}^{\ell}f(\Delta^{-\ell}g)\in L^{1}(p)$. As $f\in L^{1}(p)$ is assumed, we can use (2.18) to get

[TABLE]

where we used the first identity in (2.19) in the last line. This last expression is necessarily finite because $f,g$ and $fg$ are in $L^{1}(p)$. The general conclusion follows from the fact that any function of bounded variation is the difference between two monotone functions; the triangle inequality thus yielding the claim.

• •

For Assumption (2.28): Since $f\in L^{1}(p)$, we can apply (2.18) and the definition of $\Phi_{p}^{\ell}$ to obtain

[TABLE]

Then

[TABLE]

and

[TABLE]

Condition 1 guarantees that $L_{1}=L_{4}=0$; condition 2 guarantees that $L_{2}=L_{3}=0$. If, furthermore, $f$ is bounded then the sufficiency of 1 is immediate; if $f\in L^{2}(p)$ then it follows from the Cauchy-Schwarz inequality.

∎