The Derivative of Influence Function, Location Breakdown Point, Group Leverage and Regression Residuals' Plots

TL;DR

This paper introduces new diagnostic tools, including the influence derivative and RINFIN, to identify leverage points and influence in linear regression, especially effective in high-dimensional data.

Contribution

It develops the concept of Location Breakdown Point and RINFIN for influence diagnostics, providing theoretical insights and practical tools for regression analysis.

Findings

RINFIN effectively measures influence of data points.

High dimensionality improves detection of remote clusters.

Visual diagnostics complement RINFIN in influence detection.

Abstract

In several linear regression data sets, $Y (\in R)$ on ${\bf X} (\in R^p),$ visual comparisons of $L_1$ and $L_2$-residuals' plots indicate bad leverage cases. The phenomenon is confirmed theoretically by introducing Location Breakdown Point (LBP) of a functional $T$: any point where the derivative of $T$'s Influence Function either takes values at infinities or does not exist. Guidelines for the plots' visual comparisons as diagnostic are provided. The new tools used include E-matrix and suggest influence diagnostic RINFIN which measures the distance in the derivatives of $L_2$-residuals} at $({\bf x},y)$ from model $F$ and from gross-error model $F_{\epsilon, {\bf x},y}.$ The larger RINFIN$({\bf x},y)$ is, the larger $({\bf x},y)$'s influence in $L_2$-regression residual is. RINFIN allows measuring group influence of $k$ ${\bf x}$-neighboring data cases in a size $n$ sample using their average, $(\bar {\bf x}_k,\bar y_k),$ as one case with weight $\epsilon=k/n.$ For high dimensional, simulated data, the misclassification proportion of bad leverage cases in data's RINFIN-ordering decreases to zero as $p$ increases, thus reconfirming the blessing of high dimensionality in the detection of remote clusters. The visual diagnostic and RINFIN are successful in applications and complement each other.