StockSim: A Dual-Mode Order-Level Simulator for Evaluating Multi-Agent LLMs in Financial Markets

TL;DR

StockSim is an open-source, comprehensive simulation platform that models realistic financial market dynamics and supports diverse multi-agent LLM evaluation scenarios, incorporating real-world trading factors.

Contribution

It introduces a fully modeled, extensible simulation environment for evaluating LLMs in financial markets, including critical factors like latency and microstructure.

Findings

Supports heterogeneous trading strategies

Enables multi-agent coordination testing

Provides realistic market environment simulation

Abstract

We present StockSim, an open-source simulation platform for systematic evaluation of large language models (LLMs) in realistic financial decision-making scenarios. Unlike previous toolkits that offer limited scope, StockSim delivers a comprehensive system that fully models market dynamics and supports diverse simulation modes of varying granularity. It incorporates critical real-world factors, such as latency, slippage, and order-book microstructure, that were previously neglected, enabling more faithful and insightful assessment of LLM-based trading agents. An extensible, role-based agent framework supports heterogeneous trading strategies and multi-agent coordination, making StockSim a uniquely capable testbed for NLP research on reasoning under uncertainty and sequential decision-making. We open-source all our code at https: //github.com/harrypapa2002/StockSim.