MOT: A Mixture of Actors Reinforcement Learning Method by Optimal Transport for Algorithmic Trading

TL;DR

This paper introduces MOT, a reinforcement learning approach using multiple actors and optimal transport to adapt to different market patterns, improving trading performance and risk management.

Contribution

MOT is the first to combine disentangled multiple actors with optimal transport for market pattern modeling in RL-based trading.

Findings

MOT achieves higher profits in futures trading.

MOT balances risk and return effectively.

Ablation studies confirm component effectiveness.

Abstract

Algorithmic trading refers to executing buy and sell orders for specific assets based on automatically identified trading opportunities. Strategies based on reinforcement learning (RL) have demonstrated remarkable capabilities in addressing algorithmic trading problems. However, the trading patterns differ among market conditions due to shifted distribution data. Ignoring multiple patterns in the data will undermine the performance of RL. In this paper, we propose MOT,which designs multiple actors with disentangled representation learning to model the different patterns of the market. Furthermore, we incorporate the Optimal Transport (OT) algorithm to allocate samples to the appropriate actor by introducing a regularization loss term. Additionally, we propose Pretrain Module to facilitate imitation learning by aligning the outputs of actors with expert strategy and better balance the exploration and exploitation of RL. Experimental results on real futures market data demonstrate that MOT exhibits excellent profit capabilities while balancing risks. Ablation studies validate the effectiveness of the components of MOT.