Practical Relevance: A Formal Definition

TL;DR

This paper provides a formal, decision-theoretic definition of practical relevance in statistical analysis, emphasizing its importance alongside statistical significance and extending the concept to hypothesis testing.

Contribution

It introduces a formal, decision-theoretic framework for defining practical relevance and extends this notion to hypothesis testing contexts.

Findings

Practical relevance is linked to decision problems with actions and loss functions.

The null hypothesis can be expanded to include practically equivalent parameter values.

Involving decision theory enhances the understanding of practical relevance in analysis.

Abstract

There is a general agreement that it is important to consider the practical relevance of an effect in addition to its statistical significance, yet a formal definition of practical relevance is still pending and shall be provided within this paper. It appears that an underlying decision problem, characterized by actions and a loss function, is required to define the notion of practical relevance, rendering it a decision theoretic concept. In the context of hypothesis-based analyses, the notion of practical relevance relates to specifying the hypotheses reasonably, such that the null hypothesis does not contain only a single parameter null value, but also all parameter values that are equivalent to the null value on a practical level. In that regard, the definition of practical relevance is also extended into the context of hypotheses. The formal elaborations on the notion of practical relevance within this paper indicate that, typically, a specific decision problem is implicitly assumed when dealing with the practical relevance of an effect or some results. As a consequence, involving decision theoretic considerations into a statistical analysis suggests itself by the mere nature of the notion of practical relevance.