A simulation study to resolve conflicting evidence on the error rates from MANOVA group tests

TL;DR

This simulation study systematically evaluates the type I error rates of four MANOVA tests to clarify conflicting reports in existing literature.

Contribution

It provides a comprehensive analysis of MANOVA test error rates, resolving inconsistencies in prior research findings.

Findings

High variability in reported error rates across studies

Some tests show inflated error rates under ideal conditions

Other tests maintain nominal error rates despite assumption violations

Abstract

Popular software packages report four generalizations of the ANOVA F test when conducting a multivariate analysis of variance (MANOVA). The reported operating characteristics of these fours tests vary widely depending on which research article the reader chooses. Some studies report extremely high type I error rates for a particular test even under ideal assumptions of multivariate normality and homoskedasticity; other studies report rates near the nominal level despite violations of the model assumptions. This simulation study seeks to clarify this apparent contradiction by providing a systematic evaluation of the type I error rates of the four statistics used to test for a group effect in MANOVA.