Comparative Evaluation of Anomaly Detection Methods for Fraud Detection in Online Credit Card Payments

TL;DR

This paper evaluates various anomaly detection methods for online credit card fraud detection, comparing their effectiveness with supervised models like LightGBM, and analyzing the impact of distribution shifts on their performance.

Contribution

It provides a comprehensive comparison of AD methods and supervised learning, highlighting LightGBM's superior performance and the effects of distribution shifts in real-world fraud detection.

Findings

LightGBM outperforms AD methods across metrics

Distribution shifts negatively impact model performance

LightGBM detects most frauds identified by AD methods

Abstract

This study explores the application of anomaly detection (AD) methods in imbalanced learning tasks, focusing on fraud detection using real online credit card payment data. We assess the performance of several recent AD methods and compare their effectiveness against standard supervised learning methods. Offering evidence of distribution shift within our dataset, we analyze its impact on the tested models' performances. Our findings reveal that LightGBM exhibits significantly superior performance across all evaluated metrics but suffers more from distribution shifts than AD methods. Furthermore, our investigation reveals that LightGBM also captures the majority of frauds detected by AD methods. This observation challenges the potential benefits of ensemble methods to combine supervised, and AD approaches to enhance performance. In summary, this research provides practical insights into the utility of these techniques in real-world scenarios, showing LightGBM's superiority in fraud detection while highlighting challenges related to distribution shifts.