Small-sample performance and underlying assumptions of a bootstrap-based inference method for a general analysis of covariance model with possibly heteroskedastic and nonnormal errors

TL;DR

This paper introduces a bootstrap-based inference method for ANCOVA models that improves small-sample performance under heteroskedasticity and nonnormal errors, addressing limitations of traditional F tests.

Contribution

It combines heteroskedasticity-consistent covariance estimation with a wild bootstrap, providing a valid and practical small-sample inference method for general ANCOVA models.

Findings

The proposed method outperforms traditional F tests in small to moderate samples.

It is applicable under mild assumptions, suitable for real-world data.

Simulation results confirm improved accuracy and robustness.

Abstract

It is well known that the standard F test is severely affected by heteroskedasticity in unbalanced analysis of covariance (ANCOVA) models. Currently available potential remedies for such a scenario are based on heteroskedasticity-consistent covariance matrix estimation (HCCME). However, the HCCME approach tends to be liberal in small samples. Therefore, in the present manuscript, we propose a combination of HCCME and a wild bootstrap technique, with the aim of improving the small-sample performance. We precisely state a set of assumptions for the general ANCOVA model and discuss their practical interpretation in detail, since this issue may have been somewhat neglected in applied research so far. We prove that these assumptions are sufficient to ensure the asymptotic validity of the combined HCCME-wild bootstrap ANCOVA. The results of our simulation study indicate that our proposed test remedies the problems of the ANCOVA F test and its heteroskedasticity-consistent alternatives in small to moderate sample size scenarios. Our test only requires very mild conditions, thus being applicable in a broad range of real-life settings, as illustrated by the detailed discussion of a dataset from preclinical research on spinal cord injury. Our proposed method is ready-to-use and allows for valid hypothesis testing in frequently encountered settings (e.g., comparing group means while adjusting for baseline measurements in a randomized controlled clinical trial).