Audit Trails for Accountability in Large Language Models

TL;DR

This paper introduces a comprehensive audit trail framework for large language models, enhancing transparency and accountability by recording lifecycle events and governance actions in a tamper-evident, reviewable manner.

Contribution

It presents a lifecycle framework, a reference architecture, and an open-source implementation for creating durable audit trails in LLM workflows.

Findings

Provides a structured lifecycle framework for LLM accountability

Develops a reference architecture with lightweight emitters and audit stores

Offers an open-source Python tool for easy integration

Abstract

Large language models (LLMs) are increasingly embedded in consequential decisions across healthcare, finance, employment, and public services. Yet accountability remains fragile because process transparency is rarely recorded in a durable and reviewable form. We propose LLM audit trails as a sociotechnical mechanism for continuous accountability. An audit trail is a chronological, tamper-evident, context-rich ledger of lifecycle events and decisions that links technical provenance (models, data, training and evaluation runs, deployments, monitoring) with governance records (approvals, waivers, and attestations), so organizations can reconstruct what changed, when, and who authorized it. This paper contributes: (1) a lifecycle framework that specifies event types, required metadata, and governance rationales; (2) a reference architecture with lightweight emitters, append only audit stores, and an auditor interface supporting cross organizational traceability; and (3) a reusable, open-source Python implementation that instantiates this audit layer in LLM workflows with minimal integration effort. We conclude by discussing limitations and directions for adoption.