A General Framework on Enhancing Portfolio Management with Reinforcement Learning

TL;DR

This paper introduces a comprehensive reinforcement learning framework for portfolio management that considers practical constraints like transaction costs and short selling, comparing multiple algorithms in simulated environments.

Contribution

It presents a general RL framework for asset management incorporating real-world constraints, and evaluates three RL algorithms in a realistic simulation.

Findings

PPO outperforms other algorithms in simulated environments.

The framework effectively handles transaction costs and short selling.

Reinforcement learning can be practically applied to portfolio management.

Abstract

Portfolio management is the art and science in fiance that concerns continuous reallocation of funds and assets across financial instruments to meet the desired returns to risk profile. Deep reinforcement learning (RL) has gained increasing interest in portfolio management, where RL agents are trained base on financial data to optimize the asset reallocation process. Though there are prior efforts in trying to combine RL and portfolio management, previous works did not consider practical aspects such as transaction costs or short selling restrictions, limiting their applicability. To address these limitations, we propose a general RL framework for asset management that enables continuous asset weights, short selling and making decisions with relevant features. We compare the performance of three different RL algorithms: Policy Gradient with Actor-Critic (PGAC), Proximal Policy Optimization (PPO), and Evolution Strategies (ES) and demonstrate their advantages in a simulated environment with transaction costs. Our work aims to provide more options for utilizing RL frameworks in real-life asset management scenarios and can benefit further research in financial applications.