A General Derivative Identity for the Conditional Expectation with Focus on the Exponential Family

TL;DR

This paper derives a general derivative identity for conditional expectations, specializes it to the exponential family, and uses it to recover, generalize, and create new identities involving derivatives, variances, and cumulants.

Contribution

It introduces a unified derivative identity for conditional expectations and extends it to the exponential family, connecting derivatives with variance and cumulants, and generalizing known identities.

Findings

Derived a general Jacobian expression for conditional expectations in Markov chains.

Connected the Jacobian of conditional expectation to the conditional variance.

Established a relationship between derivatives of conditional expectations and conditional cumulants.

Abstract

Consider a pair of random vectors $(\mathbf{X},\mathbf{Y}) $ and the conditional expectation operator $\mathbb{E}[\mathbf{X}|\mathbf{Y}=\mathbf{y}]$. This work studies analytic properties of the conditional expectation by characterizing various derivative identities. The paper consists of two parts. In the first part of the paper, a general derivative identity for the conditional expectation is derived. Specifically, for the Markov chain $\mathbf{U} \leftrightarrow \mathbf{X} \leftrightarrow \mathbf{Y}$, a compact expression for the Jacobian matrix of $\mathbb{E}[\mathbf{U}|\mathbf{Y}=\mathbf{y}]$ is derived. In the second part of the paper, the main identity is specialized to the exponential family. Moreover, via various choices of the random vector $\mathbf{U}$, the new identity is used to recover and generalize several known identities and derive some new ones. As a first example, a connection between the Jacobian of $ \mathbb{E}[\mathbf{X}|\mathbf{Y}=\mathbf{y}]$ and the conditional variance is established. As a second example, a recursive expression between higher order conditional expectations is found, which is shown to lead to a generalization of the Tweedy's identity. Finally, as a third example, it is shown that the $k$-th order derivative of the conditional expectation is proportional to the $(k+1)$-th order conditional cumulant.