From specific-source feature-based to common-source score-based likelihood-ratio systems: ranking the stars

TL;DR

This paper evaluates and ranks various source-level likelihood-ratio systems for trace-reference comparison, highlighting a trade-off between performance and practical feasibility, and identifies a promising common-source feature-based approach.

Contribution

It provides a comparative analysis and ranking of LR system classes, introducing a practical, effective common-source feature-based method.

Findings

Performance varies across LR system classes with a trade-off between accuracy and feasibility.

A common-source feature-based LR system offers good performance with lower experimental demands.

All LR systems outperform using prior odds alone in updating prior odds.

Abstract

This paper studies expected performance and practical feasibility of the most commonly used classes of source-level likelihood-ratio (LR) systems when applied to a trace-reference comparison problem. The paper compares performance of these classes of LR systems (used to update prior odds) to each other and to the use of prior odds only, using strictly proper scoring rules as performance measures. It also explores practical feasibility of the classes of LR systems. The present analysis allows for a ranking of these classes of LR systems: from specific-source feature-based to common-source anchored or non-anchored score-based. A trade-off between performance and practical feasibility is observed, meaning that the best performing class of LR systems is the hardest to realise in practice, while the least performing class is the easiest to realise in practice. The other classes of LR systems are in between the two extremes. The one positive exception is a common-source feature-based LR system, with good performance and relatively low experimental demands. The paper also argues against the claim that some classes of LR systems should not be used, by showing that all systems have merit (when updating prior odds) over just using the prior odds (i.e. not using the LR system).