Detecting and adapting to crisis pattern with context based Deep Reinforcement Learning

TL;DR

This paper introduces a novel deep reinforcement learning framework with dual networks and contextual features to detect financial crises and improve portfolio management, outperforming traditional methods like Markowitz.

Contribution

The paper presents an innovative DRL approach incorporating contextual financial indicators and adversarial training for crisis detection and portfolio optimization.

Findings

Outperforms traditional portfolio optimization methods like Markowitz.

Effectively detects and anticipates financial crises such as COVID-19.

Utilizes dual networks with convolutional and LSTM layers for improved modeling.

Abstract

Deep reinforcement learning (DRL) has reached super human levels in complex tasks like game solving (Go and autonomous driving). However, it remains an open question whether DRL can reach human level in applications to financial problems and in particular in detecting pattern crisis and consequently dis-investing. In this paper, we present an innovative DRL framework consisting in two sub-networks fed respectively with portfolio strategies past performances and standard deviations as well as additional contextual features. The second sub network plays an important role as it captures dependencies with common financial indicators features like risk aversion, economic surprise index and correlations between assets that allows taking into account context based information. We compare different network architectures either using layers of convolutions to reduce network's complexity or LSTM block to capture time dependency and whether previous allocations is important in the modeling. We also use adversarial training to make the final model more robust. Results on test set show this approach substantially over-performs traditional portfolio optimization methods like Markowitz and is able to detect and anticipate crisis like the current Covid one.