MAPS: Multi-agent Reinforcement Learning-based Portfolio Management System

TL;DR

This paper introduces MAPS, a multi-agent reinforcement learning system for portfolio management that promotes diversification and adapts to market changes, outperforming baselines in US stock data.

Contribution

The paper presents a novel multi-agent RL system for portfolio management that encourages diversification and adapts to market dynamics, improving risk-adjusted returns.

Findings

MAPS outperforms baseline models in Sharpe ratio on 12-year US market data.

Adding more agents increases diversification and Sharpe ratio.

MAPS adapts to changing market conditions effectively.

Abstract

Generating an investment strategy using advanced deep learning methods in stock markets has recently been a topic of interest. Most existing deep learning methods focus on proposing an optimal model or network architecture by maximizing return. However, these models often fail to consider and adapt to the continuously changing market conditions. In this paper, we propose the Multi-Agent reinforcement learning-based Portfolio management System (MAPS). MAPS is a cooperative system in which each agent is an independent "investor" creating its own portfolio. In the training procedure, each agent is guided to act as diversely as possible while maximizing its own return with a carefully designed loss function. As a result, MAPS as a system ends up with a diversified portfolio. Experiment results with 12 years of US market data show that MAPS outperforms most of the baselines in terms of Sharpe ratio. Furthermore, our results show that adding more agents to our system would allow us to get a higher Sharpe ratio by lowering risk with a more diversified portfolio.