TransactionGPT

TL;DR

TransactionGPT is a novel foundation model utilizing a 3D-Transformer architecture to understand, generate, and predict consumer transaction data, significantly improving anomaly detection and future transaction generation.

Contribution

The paper introduces a specialized 3D-Transformer architecture for transaction data, with design innovations for modality fusion and efficiency, trained on billion-scale data, and benchmarks its performance against LLMs.

Findings

Outperforms existing models in anomaly detection

Generates more accurate future transactions

Achieves faster training and inference

Abstract

We present TransactionGPT (TGPT), a foundation model for consumer transaction data within one of the world's largest payment networks. TGPT is designed to understand and generate transaction trajectories while simultaneously supporting a variety of downstream prediction and classification tasks. We introduce a novel 3D-Transformer architecture specifically tailored for capturing the complex dynamics in payment transaction data. This architecture incorporates design innovations that enhance modality fusion and computational efficiency, while seamlessly enabling joint optimization with downstream objectives. Trained on billion-scale real-world transactions, TGPT significantly improves downstream anomaly transaction detection performance against a competitive production model and exhibits advantages over baselines in generating future transactions. We conduct extensive empirical evaluations utilizing a diverse collection of company transaction datasets spanning multiple downstream tasks, thereby enabling a thorough assessment of TGPT's effectiveness and efficiency in comparison to established methodologies. Furthermore, we examine the incorporation of LLM-derived embeddings within TGPT and benchmark its performance against fine-tuned LLMs, demonstrating that TGPT achieves superior predictive accuracy as well as faster training and inference. We anticipate that the architectural innovations and practical guidelines from this work will advance foundation models for transaction-like data and catalyze future research in this emerging field.