Defence Against the Modern Arts: the Curse of Statistics "Score-based likelihood ratios"

TL;DR

This paper critically examines score-based likelihood ratios in forensic science, highlighting their theoretical foundations, convergence properties, and limitations, to improve understanding and validation of these probabilistic tools.

Contribution

It provides a clear analysis of score-based likelihood ratios, illustrating their convergence to Bayes factors and discussing their validation and practical utility.

Findings

Score-based likelihood ratios can converge to Bayes factors.

Many score-based methods rely on ad-hoc assumptions.

Validation of these tools remains challenging.

Abstract

For several decades, legal and scientific scholars have argued that conclusions from forensic examinations should be supported by statistical data and reported within a probabilistic framework. Multiple models have been proposed to quantify the probative value of forensic evidence. Unfortunately, several of these models rely on ad-hoc strategies that are not scientifically sound. The opacity of the technical jargon used to present these models and their results, and the complexity of the techniques involved make it very difficult for the untrained user to separate the wheat from the chaff. This series of papers is intended to help forensic scientists and lawyers recognise limitations and issues in tools proposed to interpret the results of forensic examinations. This paper focuses on tools that have been proposed to leverage the use of similarity scores to assess the probative value of forensic findings. We call this family of tools "score-based likelihood ratios". In this paper, we present the fundamental concepts on which these tools are built, we describe some specific members of this family of tools, and we explore their convergence to the Bayes factor through an intuitive geometrical approach and through simulations. Finally, we discuss their validation and their potential usefulness as a decision-making tool in forensic science.