AuditCopilot: Leveraging LLMs for Fraud Detection in Double-Entry Bookkeeping

TL;DR

This paper explores the use of large language models to improve fraud detection in double-entry bookkeeping, outperforming traditional methods and enhancing interpretability through natural language explanations.

Contribution

It demonstrates that LLMs can serve as effective anomaly detectors in auditing, surpassing rule-based and classical machine learning approaches, and offers insights into AI-augmented auditing practices.

Findings

LLMs outperform traditional JETs and ML baselines in anomaly detection.

LLMs provide natural-language explanations, improving interpretability.

Results support AI-augmented auditing for better financial integrity.

Abstract

Auditors rely on Journal Entry Tests (JETs) to detect anomalies in tax-related ledger records, but rule-based methods generate overwhelming false positives and struggle with subtle irregularities. We investigate whether large language models (LLMs) can serve as anomaly detectors in double-entry bookkeeping. Benchmarking SoTA LLMs such as LLaMA and Gemma on both synthetic and real-world anonymized ledgers, we compare them against JETs and machine learning baselines. Our results show that LLMs consistently outperform traditional rule-based JETs and classical ML baselines, while also providing natural-language explanations that enhance interpretability. These results highlight the potential of \textbf{AI-augmented auditing}, where human auditors collaborate with foundation models to strengthen financial integrity.