Reinforcement Learning Applied to Trading Systems: A Survey

TL;DR

This survey reviews 29 studies applying reinforcement learning to trading, providing a structured analysis of methodologies, identifying best practices, gaps, and opportunities to guide future research in this domain.

Contribution

It offers a unified, theoretically-grounded classification and comparison of RL-based trading systems, emphasizing standards adherence and conceptual clarity.

Findings

Identified common RL formulations and design patterns in trading systems

Highlighted gaps in current methodologies and areas for further research

Provided best practice recommendations for RL application in finance

Abstract

Financial domain tasks, such as trading in market exchanges, are challenging and have long attracted researchers. The recent achievements and the consequent notoriety of Reinforcement Learning (RL) have also increased its adoption in trading tasks. RL uses a framework with well-established formal concepts, which raises its attractiveness in learning profitable trading strategies. However, RL use without due attention in the financial area can prevent new researchers from following standards or failing to adopt relevant conceptual guidelines. In this work, we embrace the seminal RL technical fundamentals, concepts, and recommendations to perform a unified, theoretically-grounded examination and comparison of previous research that could serve as a structuring guide for the field of study. A selection of twenty-nine articles was reviewed under our classification that considers RL's most common formulations and design patterns from a large volume of available studies. This classification allowed for precise inspection of the most relevant aspects regarding data input, preprocessing, state and action composition, adopted RL techniques, evaluation setups, and overall results. Our analysis approach organized around fundamental RL concepts allowed for a clear identification of current system design best practices, gaps that require further investigation, and promising research opportunities. Finally, this review attempts to promote the development of this field of study by facilitating researchers' commitment to standards adherence and helping them to avoid straying away from the RL constructs' firm ground.