Entity Extraction from High-Level Corruption Schemes via Large Language Models

TL;DR

This paper introduces a new dataset and LLM-based methods for extracting entities involved in financial crimes from news articles, demonstrating improved accuracy over existing baselines.

Contribution

It presents a novel micro-benchmark dataset for entity extraction in financial crime context and an effective LLM-based approach with disambiguation techniques.

Findings

The proposed method outperforms state-of-the-art baselines.

Prompt engineering significantly improves extraction accuracy.

Disambiguation enhances entity identification in ambiguous cases.

Abstract

The rise of financial crime that has been observed in recent years has created an increasing concern around the topic and many people, organizations and governments are more and more frequently trying to combat it. Despite the increase of interest in this area, there is a lack of specialized datasets that can be used to train and evaluate works that try to tackle those problems. This article proposes a new micro-benchmark dataset for algorithms and models that identify individuals and organizations, and their multiple writings, in news articles, and presents an approach that assists in its creation. Experimental efforts are also reported, using this dataset, to identify individuals and organizations in financial-crime-related articles using various low-billion parameter Large Language Models (LLMs). For these experiments, standard metrics (Accuracy, Precision, Recall, F1 Score) are reported and various prompt variants comprising the best practices of prompt engineering are tested. In addition, to address the problem of ambiguous entity mentions, a simple, yet effective LLM-based disambiguation method is proposed, ensuring that the evaluation aligns with reality. Finally, the proposed approach is compared against a widely used state-of-the-art open-source baseline, showing the superiority of the proposed method.