Robust exploratory mean-variance problem with drift uncertainty

TL;DR

This paper develops a robust reinforcement learning approach for the mean-variance investment problem under drift uncertainty, balancing exploration and exploitation to improve performance and risk management.

Contribution

It introduces a robust framework that incorporates model uncertainty into the exploratory mean-variance problem, emphasizing a conservative strategy that outperforms non-robust methods.

Findings

Robust strategies outperform purely exploratory ones in backtests.

The approach effectively resists downside risk in bear markets.

Robust investors focus more on exploitation under uncertainty.

Abstract

We solve a min-max problem in a robust exploratory mean-variance problem with drift uncertainty in this paper. It is verified that robust investors choose the Sharpe ratio with minimal $L^2$ norm in an admissible set. A reinforcement learning framework in the mean-variance problem provides an exploration-exploitation trade-off mechanism; if we additionally consider model uncertainty, the robust strategy essentially weights more on exploitation rather than exploration and thus reflects a more conservative optimization scheme. Finally, we use financial data to backtest the performance of the robust exploratory investment and find that the robust strategy can outperform the purely exploratory strategy and resist the downside risk in a bear market.