Adaptive Execution: Exploration and Learning of Price Impact

TL;DR

This paper introduces CTRACE, a new adaptive trading policy that learns price impact parameters while trading, achieving near-optimal regret bounds and outperforming existing methods in simulations.

Contribution

The paper proposes the CTRACE policy, a novel method for simultaneous execution and learning of price impact, with proven finite-time regret bounds and superior simulation performance.

Findings

CTRACE achieves poly-logarithmic regret bounds.

CTRACE outperforms certainty equivalent and RL algorithms in simulations.

The method effectively learns price impact while trading efficiently.

Abstract

We consider a model in which a trader aims to maximize expected risk-adjusted profit while trading a single security. In our model, each price change is a linear combination of observed factors, impact resulting from the trader's current and prior activity, and unpredictable random effects. The trader must learn coefficients of a price impact model while trading. We propose a new method for simultaneous execution and learning - the confidence-triggered regularized adaptive certainty equivalent (CTRACE) policy - and establish a poly-logarithmic finite-time expected regret bound. This bound implies that CTRACE is efficient in the sense that the ({\epsilon},{\delta})-convergence time is bounded by a polynomial function of 1/{\epsilon} and log(1/{\delta}) with high probability. In addition, we demonstrate via Monte Carlo simulation that CTRACE outperforms the certainty equivalent policy and a recently proposed reinforcement learning algorithm that is designed to explore efficiently in linear-quadratic control problems.