Evaluating Supervised Learning Models for Fraud Detection: A Comparative Study of Classical and Deep Architectures on Imbalanced Transaction Data

TL;DR

This study compares classical and deep supervised learning models for fraud detection on imbalanced transaction data, highlighting their strengths, trade-offs, and suitability for real-world deployment.

Contribution

It provides a systematic comparison of four models, including a deep GRU network, on large-scale imbalanced data, emphasizing nuanced performance metrics.

Findings

Ensemble models like Random Forest and LightGBM outperform others.

Logistic Regression offers a reliable, interpretable baseline.

GRU achieves high recall but lower precision for fraud detection.

Abstract

Fraud detection remains a critical task in high-stakes domains such as finance and e-commerce, where undetected fraudulent transactions can lead to significant economic losses. In this study, we systematically compare the performance of four supervised learning models - Logistic Regression, Random Forest, Light Gradient Boosting Machine (LightGBM), and a Gated Recurrent Unit (GRU) network - on a large-scale, highly imbalanced online transaction dataset. While ensemble methods such as Random Forest and LightGBM demonstrated superior performance in both overall and class-specific metrics, Logistic Regression offered a reliable and interpretable baseline. The GRU model showed strong recall for the minority fraud class, though at the cost of precision, highlighting a trade-off relevant for real-world deployment. Our evaluation emphasizes not only weighted averages but also per-class precision, recall, and F1-scores, providing a nuanced view of each model's effectiveness in detecting rare but consequential fraudulent activity. The findings underscore the importance of choosing models based on the specific risk tolerance and operational needs of fraud detection systems.