Understanding Structured Financial Data with LLMs: A Case Study on Fraud Detection

TL;DR

This paper introduces FinFRE-RAG, a two-stage method that enhances LLMs for fraud detection in financial data by feature reduction and retrieval-augmented learning, improving interpretability and performance.

Contribution

The paper presents a novel two-stage approach, FinFRE-RAG, that adapts LLMs for tabular fraud detection, bridging the gap between LLMs and specialized classifiers.

Findings

FinFRE-RAG significantly improves F1/MCC scores over direct prompting.

It performs competitively with strong tabular baselines in several datasets.

LLMs provide interpretable rationales, aiding fraud analysis.

Abstract

Detecting fraud in financial transactions typically relies on tabular models that demand heavy feature engineering to handle high-dimensional data and offer limited interpretability, making it difficult for humans to understand predictions. Large Language Models (LLMs), in contrast, can produce human-readable explanations and facilitate feature analysis, potentially reducing the manual workload of fraud analysts and informing system refinements. However, they perform poorly when applied directly to tabular fraud detection due to the difficulty of reasoning over many features, the extreme class imbalance, and the absence of contextual information. To bridge this gap, we introduce FinFRE-RAG, a two-stage approach that applies importance-guided feature reduction to serialize a compact subset of numeric/categorical attributes into natural language and performs retrieval-augmented in-context learning over label-aware, instance-level exemplars. Across four public fraud datasets and three families of open-weight LLMs, FinFRE-RAG substantially improves F1/MCC over direct prompting and is competitive with strong tabular baselines in several settings. Although these LLMs still lag behind specialized classifiers, they narrow the performance gap and provide interpretable rationales, highlighting their value as assistive tools in fraud analysis.