On a Nonparametric Notion of Residual and its Applications

TL;DR

This paper introduces a nonparametric residual concept for continuous random vectors that is independent of predictors, enabling a new test for conditional independence based on residual mutual independence, with practical bootstrap implementation.

Contribution

It proposes a novel nonparametric residual definition that aligns with classical residuals in normal models and develops a bootstrap-based test for conditional independence.

Findings

The residual matches classical residuals in normal regression.

The conditional independence is equivalent to residual mutual independence.

The proposed test performs well in simulations.

Abstract

Let $(X, \mathbf{Z})$ be a continuous random vector in $\mathbb{R} \times \mathbb{R}^d$, $d \ge 1$. In this paper, we define the notion of a nonparametric residual of $X$ on $\mathbf{Z}$ that is always independent of the predictor $\mathbf{Z}$. We study its properties and show that the proposed notion of residual matches with the usual residual (error) in a multivariate normal regression model. Given a random vector $(X, Y, \mathbf{Z})$ in $\mathbb{R} \times \mathbb{R} \times \mathbb{R}^d$, we use this notion of residual to show that the conditional independence between $X$ and $Y$, given $\mathbf{Z}$, is equivalent to the mutual independence of the residuals (of $X$ on $\mathbf{Z}$ and $Y$ on $\mathbf{Z}$) and $\mathbf{Z}$. This result is used to develop a test for conditional independence. We propose a bootstrap scheme to approximate the critical value of this test. We compare the proposed test, which is easily implementable, with some of the existing procedures through a simulation study.