Evidential Analysis: An Alternative to Hypothesis Testing in Normal Linear Models

TL;DR

This paper proposes an evidential analysis approach as a robust alternative to traditional hypothesis testing in normal linear models, improving interpretability and addressing common issues in scientific research.

Contribution

It introduces an evidential analysis framework for normal linear models, offering a more natural way to assess effects, evidence, and model validity compared to classical hypothesis testing.

Findings

Evidential analysis clarifies effect sizes and evidence strength.

Application to 2-way ANOVA demonstrates practical utility.

Addresses limitations of null hypothesis significance testing.

Abstract

Statistical hypothesis testing, as formalized by 20th Century statisticians and taught in college statistics courses, has been a cornerstone of 100 years of scientific progress. Nevertheless, the methodology is increasingly questioned in many scientific disciplines. We demonstrate in this paper how many of the worrisome aspects of statistical hypothesis testing can be ameliorated with concepts and methods from evidential analysis. The model family we treat is the familiar normal linear model with fixed effects, embracing multiple regression and analysis of variance, a warhorse of everyday science in labs and field stations. Questions about study design, the applicability of the null hypothesis, the effect size, error probabilities, evidence strength, and model misspecification become more naturally housed in an evidential setting. We provide a completely worked example featuring a 2-way analysis of variance.