Hierarchical Decision Ensembles- An inferential framework for uncertain Human-AI collaboration in forensic examinations

TL;DR

This paper introduces a hierarchical decision ensemble framework that enhances trust and interpretability of complex statistical models in forensic human-AI collaboration, improving validation and acceptance of evidence analysis.

Contribution

It presents a novel inferential framework that bridges forensic domain expertise and statistical model outputs, facilitating better validation and trust in forensic evidence assessment.

Findings

Framework improves examiner trust in statistical models

Enables validation of model claims by forensic experts

Facilitates critical assessment of complex forensic results

Abstract

Forensic examination of evidence like firearms and toolmarks, traditionally involves a visual and therefore subjective assessment of similarity of two questioned items. Statistical models are used to overcome this subjectivity and allow specification of error rates. These models are generally quite complex and produce abstract results at different levels of the analysis. Presenting such metrics and complicated results to examiners is challenging, as examiners generally do not have substantial statistical training to accurately interpret results. This creates distrust in statistical modelling and lowers the rate of acceptance of more objective measures that the discipline at large is striving for. We present an inferential framework for assessing the model and its output. The framework is designed to calibrate trust in forensic experts by bridging the gap between domain specific knowledge and predictive model results, allowing forensic examiners to validate the claims of the predictive model while critically assessing results.