Regret-Optimized Portfolio Enhancement through Deep Reinforcement Learning and Future Looking Rewards

TL;DR

This paper presents a deep reinforcement learning approach using PPO and future-looking rewards to enhance traditional portfolio strategies, focusing on dynamic rebalancing and transaction cost management to improve returns and reduce drawdowns.

Contribution

It introduces a novel agent-based method that improves existing portfolio strategies with a regret-optimized reward function and synthetic data training for better generalization.

Findings

Enhanced portfolio return and reduced maximum drawdown.

Effective rebalancing strategy outperforming baselines.

Robust performance across stochastic market conditions.

Abstract

This paper introduces a novel agent-based approach for enhancing existing portfolio strategies using Proximal Policy Optimization (PPO). Rather than focusing solely on traditional portfolio construction, our approach aims to improve an already high-performing strategy through dynamic rebalancing driven by PPO and Oracle agents. Our target is to enhance the traditional 60/40 benchmark (60% stocks, 40% bonds) by employing the Regret-based Sharpe reward function. To address the impact of transaction fee frictions and prevent signal loss, we develop a transaction cost scheduler. We introduce a future-looking reward function and employ synthetic data training through a circular block bootstrap method to facilitate the learning of generalizable allocation strategies. We focus on two key evaluation measures: return and maximum drawdown. Given the high stochasticity of financial markets, we train 20 independent agents each period and evaluate their average performance against the benchmark. Our method not only enhances the performance of the existing portfolio strategy through strategic rebalancing but also demonstrates strong results compared to other baselines.