Deep Reinforcement Learning for Online Optimal Execution Strategies

TL;DR

This paper presents a novel deep reinforcement learning algorithm based on DDPG for learning optimal execution strategies in financial markets, effectively handling transient price impacts and adapting to market changes.

Contribution

Introduces a new actor-critic DDPG-based algorithm for non-Markovian optimal execution, addressing transient price impacts and market variability.

Findings

Successfully approximates optimal execution strategies

Demonstrates adaptability to changing market parameters

Reduces reliance on human intervention

Abstract

This paper tackles the challenge of learning non-Markovian optimal execution strategies in dynamic financial markets. We introduce a novel actor-critic algorithm based on Deep Deterministic Policy Gradient (DDPG) to address this issue, with a focus on transient price impact modeled by a general decay kernel. Through numerical experiments with various decay kernels, we show that our algorithm successfully approximates the optimal execution strategy. Additionally, the proposed algorithm demonstrates adaptability to evolving market conditions, where parameters fluctuate over time. Our findings also show that modern reinforcement learning algorithms can provide a solution that reduces the need for frequent and inefficient human intervention in optimal execution tasks.