Time-uniform confidence bands for the CDF under nonstationarity

TL;DR

This paper develops valid, time-uniform confidence bands for the cumulative distribution function (CDF) of a random variable in nonstationary settings, extending to importance-weighted cases for counterfactual analysis.

Contribution

It introduces the first computationally feasible, always valid confidence bounds on the CDF under nonstationarity, with convergence guarantees and applicability to importance-weighted data.

Findings

Provides time-uniform confidence bands valid under nonstationarity.

Extends bounds to importance-weighted estimations for counterfactual distributions.

Guarantees convergence in arbitrary data-dependent environments.

Abstract

Estimation of the complete distribution of a random variable is a useful primitive for both manual and automated decision making. This problem has received extensive attention in the i.i.d. setting, but the arbitrary data dependent setting remains largely unaddressed. Consistent with known impossibility results, we present computationally felicitous time-uniform and value-uniform bounds on the CDF of the running averaged conditional distribution of a real-valued random variable which are always valid and sometimes trivial, along with an instance-dependent convergence guarantee. The importance-weighted extension is appropriate for estimating complete counterfactual distributions of rewards given controlled experimentation data exhaust, e.g., from an A/B test or a contextual bandit.

Tables

Table 1. Table 1: Comparison to prior art for CDF estimation. See Section   5 for details.

Reference	Quantile-
	Time-
	Non-
	Non-
	Non-
	Counter-
	$w_{\max }$-
	free?111$w_{\max }$ free techniques are valid with unbounded importance weights.
HR22	✓	✓		✓	✓		N/A
HLLA21	✓			✓	✓	✓
UnO21 IID	✓			✓	✓	✓	✓
UnO21 NS	✓		✓			✓	✓
WS22 §4	✓	✓		✓	✓	✓	✓
this paper	✓	✓	✓	✓	✓	✓	✓

Table 2. Table 2 : Comparison of DDRM and Empirical Bernstein on i.i.d. X t ∼ Beta ​ ( 6 , 3 ) similar-to subscript 𝑋 𝑡 Beta 6 3 X_{t}\sim\text{Beta}(6,3) , for different W t subscript 𝑊 𝑡 W_{t} . Width denotes the maximum bound width sup v U t ​ ( v ) − L t ​ ( v ) subscript supremum 𝑣 subscript 𝑈 𝑡 𝑣 subscript 𝐿 𝑡 𝑣 \sup_{v}U_{t}(v)-L_{t}(v) . Time is for computing the bound at 1000 equally spaced points.

$W_t$	What	Width	Time (sec)
$\text{Exp}\left(1\right)$	DDRM	0.09	24.8
	Emp. Bern	0.10	1.0
$\text{Pareto}\left(3/2\right)$	DDRM	0.052	59.4
	Emp. Bern	0.125	2.4

Equations

$${\overline{\mathrm{CDF}}}_t(v)\doteq \frac{1}{t}\sum _{s\le t}{\mathbb{E}}_{s-1}\left[1_{X_s\le v}\right],$$

$$\mathbb{P}\left(\begin{array}{c}\forall t\in \mathbb{N}\\ \forall v\in \mathbb{R}\end{array}:L_t(v)\le {\overline{\mathrm{CDF}}}_t(v)\le U_t(v)\right)\ge 1-2\alpha .$$

$$E_t(\lambda )\doteq \exp \left(\lambda S_t-\sum _{s\le t}\log \left(h(\lambda ,{\theta }_s)\right)\right),$$

$$P(\forall t:{\overline{\mathrm{CDF}}}_t(v)\ge {\Lambda }_t(\rho ;\delta ,{\Psi }_t))$$

$$P(\forall t:{\overline{\mathrm{CDF}}}_t(v)\le {\Xi }_t(\rho ;\delta ,{\Psi }_t))$$

$$\begin{array}{rl}& \forall t,\forall v:U_t(v)-L_t(v)\le \\ & \sqrt{\frac{V_t}{t}}+\tilde{O}\left(\sqrt{\frac{V_t}{t}\log \left({\xi }_t^{-2}{\alpha }^{-1}{t}^{3/2}\right)}\right),\end{array}$$

$$\begin{array}{rl}& \forall t,\forall v:U_t(v)-L_t(v)\le \\ & B_t+\sqrt{\frac{(\tau +V_t)/t}{t}}\\ & +\tilde{O}\left(\sqrt{\frac{(\tau +V_t)/t}{t}\log \left({\xi }_t^{-2}{\alpha }^{-1}\right)}\right)\\ & +\tilde{O}(t^{-1}\log \left({\xi }_t^{-2}{\alpha }^{-1}\right)),\end{array}$$

$$\mathbb{P}(\forall x\in \mathbb{R},{\dot{L}}_n(x)\le F(x)\le {\dot{U}}_n(x))\ge 1-\alpha ,$$

$$\mathbb{P}(\forall x\in \mathbb{R},t\in \mathbb{N},\text{widebar}{L}_t(x)\le F(x)\le \text{widebar}{U}_t(x))\ge 1-\alpha .$$

$$E_t(\lambda )\doteq \exp \left(\lambda S_t-\sum _{s\le t}\log \left(h(\lambda ,{\theta }_s)\right)\right),$$

$$E_t(\lambda )\ge \exp \left(\lambda t\left(q_t-{\hat{q}}_t\right)-th\left(\lambda ,q_t\right)\right),$$

$$P(\exists t:{\overline{\mathrm{CDF}}}_t(v)>{\Xi }_t(i/ϵ(d);{\delta }_d,d,{\Psi }_t))$$

$$P\left(\exists d\in \mathbb{N},i\in \{1,\dots ,d\},t\in \mathbb{N}:{\overline{\mathrm{CDF}}}_t(i/ϵ(d))>{\Xi }_t(i/ϵ(d);\delta ,d,{\Psi }_t)\right)\le \alpha .$$

$$P(\forall t,d:{\overline{\mathrm{CDF}}}_t(v)\le {\overline{\mathrm{CDF}}}_t({\rho }_d)\le {\Xi }_t({\rho }_d;{\delta }_d,d,{\Psi }_t))\ge 1-\alpha .$$

$$M(t;q_t,\tau )$$

$$\frac{\alpha }{{\eta }^{2d}}$$

$${\overline{\mathrm{CDF}}}_t(ϵ(d)\lceil ϵ(d{)}^{-1}v\rceil )-{\overline{\mathrm{CDF}}}_t(v)\le 1/({\xi }_t{\eta }^d),$$

$$r_d(t)$$

$$|k_d|-1$$

$${\xi }_t/\sqrt{{\psi }_t}$$

$$⟹1+|k_d|$$

$$⟹r_d(t)$$

$$E_t(\lambda )\ge \exp \left(\lambda \left(\min \left(t,\sum _{s\le t}{Y}_s\right)-{\mathbb{E}}_{s-1}\left[Y_s\right]\right)+\sum _{s\le t}\log \left(1+\lambda \left(Y_s-\overline{Y_t}\right)\right)-\text{Reg}(t)\right),$$

$$M_t^{\text{EB}}\doteq \left(\frac{{\tau }^{\tau }{e}^{-\tau }}{\Gamma (\tau )-\Gamma (\tau ,\tau )}\right)\left(\frac{1}{\tau +V_t}\right){}_1{F}_1\left(1,V_t+\tau +1,S_t+V_t+\tau \right),$$

$$\frac{\alpha }{{\eta }^{2d}}$$

$$u\left(V_t;\tau ,\frac{\alpha }{{\eta }^{2d}}\right)$$

$$+\frac{1}{t}\log \left(\sqrt{\frac{\tau +V_t}{2\pi }}{e}^{-\frac{1}{12(\tau +V_t)+1}}\left(\frac{1+{\eta }^{2d}{\alpha }^{-1}}{C(\tau )}\right)\right),$$

$${\overline{\mathrm{CDF}}}_t(ϵ(d)\lceil ϵ(d{)}^{-1}v\rceil )-{\overline{\mathrm{CDF}}}_t(v)\le 1/({\xi }_t{\eta }^d),$$

$$r_d(t)$$

$$=\sqrt{\frac{(\tau +V_t)/t}{t}}+\tilde{O}\left(\sqrt{\frac{(\tau +V_t)/t}{t}\log \left({\xi }_t^{-2}{\alpha }^{-1}\right)}\right)+\tilde{O}(t^{-1}\log \left({\xi }_t^{-2}{\alpha }^{-1}\right)),$$

$$\exp \left(\lambda S_t-{\psi }_e(\lambda )V_t\right),$$

$${\psi }_e(\lambda )$$

Full text

1 Introduction

What would have happened if I had acted differently? Although this question is as old as time itself, successful companies have recently embraced this question via counterfactual estimation of outcomes from the exhaust of their controlled experimentation platforms, e.g., based upon A/B testing or contextual bandits. These experiments are run in the real (digital) world, which is rich enough to demand statistical techniques that are non-asymptotic, non-parametric, and non-stationary. Although recent advances admit characterizing counterfactual average outcomes in this general setting, counterfactually estimating a complete distribution of outcomes is heretofore only possible with additional assumptions. Nonethless, the practical importance of this problem has motivated multiple solutions: see Section 1 for a summary, and Section 5 for complete discussion.

Intriguingly, this problem is provably impossible in the data dependent setting without additional assumptions. Rakhlin et al. (2015) Consequently, our bounds always achieve non-asymptotic coverage, but may converge to zero width slowly or not at all, depending on the hardness of the instance. We call this design principle AVAST (Always Valid And Sometimes Trivial).

In pursuit of our ultimate goal, we derive factual distribution estimators which are useful for estimating the complete distribution of outcomes from direct experience.

Contributions

In Section 3.1 we provide a time and value uniform upper bound on the CDF of the averaged historical conditional distribution of a discrete-time real-valued random process. Consistent with the lack of sequential uniform convergence of linear threshold functions (Rakhlin et al., 2015), the bounds are always valid and sometimes trivial, but with an instance-dependent guarantee: when the data generating process is smooth qua Block et al. (2022) with respect to the uniform distribution on the unit interval, the bound width adapts to the unknown smoothness parameter. 2. 2.

In Section 3.2 we extend the previous technique to distributions with support over the entire real line, and further to distributions with a known countably infinite or unknown nowhere dense set of discrete jumps; with analogous instance-dependent guarantees. 3. 3.

In Section 3.3 we extend the previous techniques to importance-weighted random variables, achieving our ultimate goal of estimating a complete counterfactual distribution of outcomes.

We exhibit our techniques in various simulations in Section 4. Computationally our procedures have comparable cost to point estimation of the empirical CDF, as the empirical CDF is a sufficient statistic.