LOB-Bench: Benchmarking Generative AI for Finance -- an Application to Limit Order Book Data

TL;DR

LOB-Bench introduces a comprehensive benchmarking framework for evaluating generative models of limit order book data, addressing the challenge of quantitative assessment in noisy, high-dimensional financial sequences.

Contribution

This paper presents LOB-Bench, a new Python-based benchmark with diverse statistical and market impact metrics for assessing generative models of LOB data, including autoregressive GenAI methods.

Findings

Autoregressive GenAI models outperform traditional LOB models.

The benchmark supports multivariate statistical evaluation and market impact analysis.

Generative models show promising realism in simulating LOB data.

Abstract

While financial data presents one of the most challenging and interesting sequence modelling tasks due to high noise, heavy tails, and strategic interactions, progress in this area has been hindered by the lack of consensus on quantitative evaluation paradigms. To address this, we present LOB-Bench, a benchmark, implemented in python, designed to evaluate the quality and realism of generative message-by-order data for limit order books (LOB) in the LOBSTER format. Our framework measures distributional differences in conditional and unconditional statistics between generated and real LOB data, supporting flexible multivariate statistical evaluation. The benchmark also includes features commonly used LOB statistics such as spread, order book volumes, order imbalance, and message inter-arrival times, along with scores from a trained discriminator network. Lastly, LOB-Bench contains "market impact metrics", i.e. the cross-correlations and price response functions for specific events in the data. We benchmark generative autoregressive state-space models, a (C)GAN, as well as a parametric LOB model and find that the autoregressive GenAI approach beats traditional model classes.