On Testing Independence and Goodness-of-fit in Linear Models

TL;DR

This paper introduces a new omnibus test for checking independence and goodness-of-fit in linear models, using the Hilbert--Schmidt independence criterion, which is simple, consistent, and outperforms existing methods in power.

Contribution

The paper proposes a novel, easy-to-compute omnibus test based on the Hilbert--Schmidt independence criterion for linear models, with proven consistency and improved power over competitors.

Findings

The test effectively detects violations of independence and model fit.

Simulation shows superior power compared to existing methods.

Real data analysis demonstrates practical usefulness.

Abstract

We consider a linear regression model and propose an omnibus test to simultaneously check the assumption of independence between the error and the predictor variables, and the goodness-of-fit of the parametric model. Our approach is based on testing for independence between the residual obtained from the parametric fit and the predictor using the Hilbert--Schmidt independence criterion (Gretton et al. (2008)). The proposed method requires no user-defined regularization, is simple to compute, based merely on pairwise distances between points in the sample, and is consistent against all alternatives. We develop distribution theory for the proposed test statistic, both under the null and the alternative hypotheses, and devise a bootstrap scheme to approximate its null distribution. We prove the consistency of the bootstrap scheme. A simulation study shows that our method has better power than its main competitors. Two real datasets are analyzed to demonstrate the scope and usefulness of our method.