Advancing Anomaly Detection: Non-Semantic Financial Data Encoding with LLMs

TL;DR

This paper explores using Large Language Model embeddings to improve anomaly detection in financial ledger data, demonstrating that LLMs enhance model performance especially with sparse, non-semantic features.

Contribution

Introduces a novel approach employing LLM embeddings for non-semantic financial data, improving anomaly detection accuracy over traditional methods.

Findings

LLM embeddings improve anomaly detection performance.

Models outperform baselines, sometimes significantly.

LLMs help address feature sparsity in financial data.

Abstract

Detecting anomalies in general ledger data is of utmost importance to ensure trustworthiness of financial records. Financial audits increasingly rely on machine learning (ML) algorithms to identify irregular or potentially fraudulent journal entries, each characterized by a varying number of transactions. In machine learning, heterogeneity in feature dimensions adds significant complexity to data analysis. In this paper, we introduce a novel approach to anomaly detection in financial data using Large Language Models (LLMs) embeddings. To encode non-semantic categorical data from real-world financial records, we tested 3 pre-trained general purpose sentence-transformer models. For the downstream classification task, we implemented and evaluated 5 optimized ML models including Logistic Regression, Random Forest, Gradient Boosting Machines, Support Vector Machines, and Neural Networks. Our experiments demonstrate that LLMs contribute valuable information to anomaly detection as our models outperform the baselines, in selected settings even by a large margin. The findings further underscore the effectiveness of LLMs in enhancing anomaly detection in financial journal entries, particularly by tackling feature sparsity. We discuss a promising perspective on using LLM embeddings for non-semantic data in the financial context and beyond.