Evaluating the Reliability of Digital Forensic Evidence Discovered by Large Language Model: A Case Study

TL;DR

This paper presents a structured framework utilizing large language models and a digital forensic knowledge graph to automate artifact extraction, validate evidence, and improve the reliability of AI-identified digital forensic evidence.

Contribution

The paper introduces a novel, scalable framework combining LLM analysis and a forensic knowledge graph to enhance the credibility and traceability of AI-derived digital evidence.

Findings

Achieved over 95% accuracy in artifact extraction.

Ensured strong chain-of-custody support.

Maintained robust contextual consistency in forensic relationships.

Abstract

The growing reliance on AI-identified digital evidence raises significant concerns about its reliability, particularly as large language models (LLMs) are increasingly integrated into forensic investigations. This paper proposes a structured framework that automates forensic artifact extraction, refines data through LLM-driven analysis, and validates results using a Digital Forensic Knowledge Graph (DFKG). Evaluated on a 13 GB forensic image dataset containing 61 applications, 2,864 databases, and 5,870 tables, the framework ensures artifact traceability and evidentiary consistency through deterministic Unique Identifiers (UIDs) and forensic cross-referencing. We propose this methodology to address challenges in ensuring the credibility and forensic integrity of AI-identified evidence, reducing classification errors, and advancing scalable, auditable methodologies. A comprehensive case study on this dataset demonstrates the framework's effectiveness, achieving over 95 percent accuracy in artifact extraction, strong support of chain-of-custody adherence, and robust contextual consistency in forensic relationships. Key results validate the framework's ability to enhance reliability, reduce errors, and establish a legally sound paradigm for AI-assisted digital forensics.