Which Trading Agent is Best? Using a Threaded Parallel Simulation of a Financial Market Changes the Pecking-Order

TL;DR

This paper introduces a threaded simulation model for financial markets that reveals the performance hierarchy of trading algorithms changes significantly when parallelism is modeled more realistically, challenging previous conclusions.

Contribution

The paper develops Threaded-BSE, an extended market simulator that models asynchronous parallel trading, and demonstrates its impact on the evaluation of trading algorithms compared to simpler models.

Findings

Parallel simulation alters the dominance hierarchy of trading algorithms.

Simple minimal simulations may not reliably predict real market performance.

Realistic asynchronous modeling is crucial for accurate algorithm assessment.

Abstract

This paper presents novel results generated from a new simulation model of a contemporary financial market, that cast serious doubt on the previously widely accepted view of the relative performance of various well-known public-domain automated-trading algorithms. Various public-domain trading algorithms have been proposed over the past 25 years in a kind of arms-race, where each new trading algorithm was compared to the previous best, thereby establishing a "pecking order", i.e. a partially-ordered dominance hierarchy from best to worst of the various trading algorithms. Many of these algorithms were developed and tested using simple minimal simulations of financial markets that only weakly approximated the fact that real markets involve many different trading systems operating asynchronously and in parallel. In this paper we use BSE, a public-domain market simulator, to run a set of experiments generating benchmark results from several well-known trading algorithms. BSE incorporates a very simple time-sliced approach to simulating parallelism, which has obvious known weaknesses. We then alter and extend BSE to make it threaded, so that different trader algorithms operate asynchronously and in parallel: we call this simulator Threaded-BSE (TBSE). We then re-run the trader experiments on TBSE and compare the TBSE results to our earlier benchmark results from BSE. Our comparison shows that the dominance hierarchy in our more realistic experiments is different from the one given by the original simple simulator. We conclude that simulated parallelism matters a lot, and that earlier results from simple simulations comparing different trader algorithms are no longer to be entirely trusted.