Stop the tests: Opinion bias and statistical tests

TL;DR

The paper argues that hypothesis testing is fundamentally flawed due to cognitive biases and the lack of logical justification, suggesting that testing should be abandoned in hypothesis evaluation.

Contribution

It highlights the influence of opinion bias on hypothesis testing and advocates for discarding testing procedures in favor of unbiased reasoning.

Findings

Cognitive biases affect hypothesis selection

Logic cannot definitively prove ideas right or wrong

Testing procedures are prone to bias and error

Abstract

When statisticians quarrel about hypothesis testing, the debate usually focus on which method is the correct one. The fundamental question of whether we should test hypothesis at all tends to be forgotten. This lack of debate has its roots on our desire to have ideas we believe and defend. But cognitive experiments have been showing that, when we do choose ideas, we become prey to a large number of biases. Several of our biases can be grouped together in a single description, an opinion bias. This opinion bias is nothing more than our desire to believe in something and to defend it. Also, despite our feelings, believing has no solid logical or philosophical grounds. In this paper, I will show that if we combine the fact that even logic can never prove an idea right or wrong and the problems our brains cause when we pick ideas, hypothesis testing and its terminology are a recipe for disaster. Testing should have no place when we are thinking about hypothesis.