Considerations for missing data, outliers and transformations in permutation testing for ANOVA, ASCA(+) and related factorizations

TL;DR

This paper examines how missing data, outliers, and non-normal distributions impact permutation testing in ANOVA, ASCA(+), and related factorizations, offering practical guidelines for robust analysis.

Contribution

It provides a detailed analysis of issues affecting permutation-based inference in factorial designs and proposes best practices to improve robustness against data irregularities.

Findings

Missing data and outliers significantly affect Type I and II error rates.

Normality departures can bias permutation test results.

Recommended data preprocessing strategies enhance analysis reliability.

Abstract

Multifactorial experimental designs allow us to assess the contribution of several factors, and potentially their interactions, to one or several responses of interests. Following the principles of the partition of the variance advocated by Sir R.A. Fisher, the experimental responses are factored into the quantitative contribution of main factors and interactions. A popular approach to perform this factorization in both ANOVA and ASCA(+) is through General Linear Models. Subsequently, different inferential approaches can be used to identify whether the contributions are statistically significant or not. Unfortunately, the performance of inferential approaches in terms of Type I and Type II errors can be heavily affected by missing data, outliers and/or the departure from normality of the distribution of the responses, which are commonplace problems in modern analytical experiments. In this paper, we study these problem and suggest good practices of application.