Exploratory Mean-Variance with Jumps: An Equilibrium Approach

TL;DR

This paper develops a reinforcement learning-based equilibrium approach for mean-variance portfolio optimization with jump-diffusion market dynamics, addressing time-inconsistency and demonstrating practical profitability on real market data.

Contribution

It introduces a novel RL framework for mean-variance optimization with jumps, incorporating time-inconsistent preferences and deriving an equilibrium policy.

Findings

RL model converges to true parameters in simulations

Proposed approach is profitable in most real market tests

Accounts for jumps and time-inconsistency in portfolio optimization

Abstract

Revisiting the continuous-time Mean-Variance (MV) Portfolio Optimization problem, we model the market dynamics with a jump-diffusion process and apply Reinforcement Learning (RL) techniques to facilitate informed exploration within the control space. We recognize the time-inconsistency of the MV problem and adopt the time-inconsistent control (TIC) approach to analytically solve for an exploratory equilibrium investment policy, which is a Gaussian distribution centered on the equilibrium control of the classical MV problem. Our approach accounts for time-inconsistent preferences and actions, and our equilibrium policy is the best option an investor can take at any given time during the investment period. Moreover, we leverage the martingale properties of the equilibrium policy, design a RL model, and propose an Actor-Critic RL algorithm. All of our RL model parameters converge to the corresponding true values in a simulation study. Our numerical study on 24 years of real market data shows that the proposed RL model is profitable in 13 out of 14 tests, demonstrating its practical applicability in real world investment.