DynBERG: Dynamic BERT-based Graph neural network for financial fraud detection

TL;DR

DynBERG is a novel graph neural network that combines Graph-BERT with GRU to effectively detect financial fraud in dynamic, directed transaction networks, outperforming existing methods especially during market shifts.

Contribution

The paper introduces DynBERG, a new architecture integrating Graph-BERT with GRU for dynamic, directed graphs, tailored for financial fraud detection in evolving cryptocurrency networks.

Findings

DynBERG outperforms EvolveGCN before market shutdown.

DynBERG surpasses GCN after the market event.

Incorporating GRU improves temporal modeling of transactions.

Abstract

Financial fraud detection is critical for maintaining the integrity of financial systems, particularly in decentralised environments such as cryptocurrency networks. Although Graph Convolutional Networks (GCNs) are widely used for financial fraud detection, graph Transformer models such as Graph-BERT are gaining prominence due to their Transformer-based architecture, which mitigates issues such as over-smoothing. Graph-BERT is designed for static graphs and primarily evaluated on citation networks with undirected edges. However, financial transaction networks are inherently dynamic, with evolving structures and directed edges representing the flow of money. To address these challenges, we introduce DynBERG, a novel architecture that integrates Graph-BERT with a Gated Recurrent Unit (GRU) layer to capture temporal evolution over multiple time steps. Additionally, we modify the underlying algorithm to support directed edges, making DynBERG well-suited for dynamic financial transaction analysis. We evaluate our model on the Elliptic dataset, which includes Bitcoin transactions, including all transactions during a major cryptocurrency market event, the Dark Market Shutdown. By assessing DynBERG's resilience before and after this event, we analyse its ability to adapt to significant market shifts that impact transaction behaviours. Our model is benchmarked against state-of-the-art dynamic graph classification approaches, such as EvolveGCN and GCN, demonstrating superior performance, outperforming EvolveGCN before the market shutdown and surpassing GCN after the event. Additionally, an ablation study highlights the critical role of incorporating a time-series deep learning component, showcasing the effectiveness of GRU in modelling the temporal dynamics of financial transactions.