Open Banking Foundational Model: Learning Language Representations from Few Financial Transactions

TL;DR

This paper presents a multimodal foundational model for financial transactions that combines structured and unstructured data, improving performance especially in data-scarce Open Banking scenarios and demonstrating cross-institutional generalization.

Contribution

It introduces a novel multimodal, self-supervised model for financial transaction representation, pioneering large-scale cross-institutional analysis in North America.

Findings

Outperforms classical feature engineering methods

Effective in data-scarce Open Banking scenarios

Generalizes across different financial institutions

Abstract

We introduced a multimodal foundational model for financial transactions that integrates both structured attributes and unstructured textual descriptions into a unified representation. By adapting masked language modeling to transaction sequences, we demonstrated that our approach not only outperforms classical feature engineering and discrete event sequence methods but is also particularly effective in data-scarce Open Banking scenarios. To our knowledge, this is the first large-scale study across thousands of financial institutions in North America, providing evidence that multimodal representations can generalize across geographies and institutions. These results highlight the potential of self-supervised models to advance financial applications ranging from fraud prevention and credit risk to customer insights