Testing replication for an agent-based model of market fragmentation and latency arbitrage

TL;DR

This paper attempts to replicate Wah and Wellman's 2016 market model, highlighting challenges due to missing details, and proposes methods to support rigorous quantitative comparison and extension.

Contribution

It demonstrates the difficulties of replication due to incomplete details, introduces bootstrap confidence intervals for analysis, and provides an ODD protocol for future work.

Findings

Replication is hindered by missing implementation details.

Bootstrap confidence intervals help support quantitative tests.

Market fragmentation effects are sensitive to trader strategy assumptions.

Abstract

This study strengthens the foundations of multi-venue market modeling by attempting an independent replication of Wah and Wellman's 2016 model of latency arbitrage in a fragmented market. We find that faithful replication is hindered by missing implementation details in the original paper and limited quantitative reporting. We demonstrate that increasing the number of simulation runs beyond the original design allows for the creation of bootstrap confidence intervals to support rigorous tests of quantitative alignment, compensating for lacking distributional information (e.g. variance). We also demonstrate that increased complexity across the modeled scenarios corresponds with increased difficulty aligning to the original results. We draw on a codebase released by the original authors in connection with a later paper to recover additional implementation details; however, we reject quantitative alignment between that codebase and the published results. Combining information from the paper and the released code, we achieve relational equivalence for most metrics but reject quantitative alignment for model settings where latency is non-zero. We show that many of the qualitative takeaways from the original paper on the effects of market fragmentation and latency arbitrage are sensitive to the specifics of a `greedy strategy' extension given to the zero-intelligence (ZI) trader agents. Under an alternative interpretation of this strategy, we find that market fragmentation decreases execution times in all experiments and increases trader welfare in most experiments. Finally, to facilitate future replication, critique, and extension, we provide an ODD (Overview, Design concepts, Details) protocol for our implementations of the model.