How to Measure Evidence and Its Strength: Bayes Factors or Relative Belief Ratios?

TL;DR

This paper compares Bayes factors and relative belief ratios as measures of statistical evidence, arguing that the latter has better properties and proposing a more accurate way to measure evidence strength.

Contribution

It demonstrates the advantages of relative belief ratios over Bayes factors and proposes a new method for measuring evidence strength.

Findings

Relative belief ratios have better properties as evidence measures.

The Bayes factor equals the relative belief ratio under certain restrictions.

Using the size of the Bayes factor to measure evidence strength is inadequate.

Abstract

Both the Bayes factor and the relative belief ratio satisfy the principle of evidence and so can be seen to be valid measures of statistical evidence. Certainly Bayes factors are regularly employed. The question then is: which of these measures of evidence is more appropriate? It is argued here that there are questions concerning the validity of a current commonly used definition of the Bayes factor based on a mixture prior and, when all is considered, the relative belief ratio has better properties as a measure of evidence. It is further shown that, when a natural restriction on the mixture prior is imposed, the Bayes factor equals the relative belief ratio obtained without using the mixture prior. Even with this restriction, this still leaves open the question of how the strength of evidence is to be measured. It is argued here that the current practice of using the size of the Bayes factor to measure strength is not correct and a solution to this issue is presented. Several general criticisms of these measures of evidence are also discussed and addressed.