Statistical methods: Basic concepts, interpretations, and cautions

TL;DR

This paper discusses the variability and controversy in statistical methods for studying associations and causality, emphasizing the importance of understanding assumptions and limitations in statistical inference.

Contribution

It proposes an approach that grounds statistical models in data description and treats inferences as assumptions-based speculations, highlighting the need for cautious interpretation.

Findings

Models should be grounded in data description

Inferences are based on assumptions that cannot be fully validated

Highlights limitations of significance and confidence measures

Abstract

The study of associations and their causal explanations is a central research activity whose methodology varies tremendously across fields. Even within specialized subfields, comparisons across textbooks and journals reveals that the basics are subject to considerable variation and controversy. This variation is often obscured by the singular viewpoints presented within textbooks and journal guidelines, which may be deceptively written as if the norms they adopt are unchallenged. Furthermore, human limitations and the vastness within fields imply that no one can have expertise across all subfields and that interpretations will be severely constrained by the limitations of studies of human populations. The present chapter outlines an approach to statistical methods that attempts to recognize these problems from the start, rather than assume they are absent as in the claims of 'statistical significance' and 'confidence' ordinarily attached to statistical tests and interval estimates. It does so by grounding models and statistics in data description, and treating inferences from them as speculations based on assumptions that cannot be fully validated or checked using the analysis data.