Commodities Trading through Deep Policy Gradient Methods

TL;DR

This paper explores deep reinforcement learning for commodities trading, introducing novel algorithms and time-discretization schemes that adapt to market volatility, resulting in significantly improved trading performance and customizable risk profiles.

Contribution

It presents new DRL algorithms with adaptive time discretization and demonstrates their effectiveness in commodities trading, outperforming traditional methods.

Findings

DRL models increase Sharpe ratio by 83% over buy-and-hold.

Actor-based models outperform actor-critic models.

CNN-based models slightly outperform LSTM-based models.

Abstract

Algorithmic trading has gained attention due to its potential for generating superior returns. This paper investigates the effectiveness of deep reinforcement learning (DRL) methods in algorithmic commodities trading. It formulates the commodities trading problem as a continuous, discrete-time stochastic dynamical system. The proposed system employs a novel time-discretization scheme that adapts to market volatility, enhancing the statistical properties of subsampled financial time series. To optimize transaction-cost- and risk-sensitive trading agents, two policy gradient algorithms, namely actor-based and actor-critic-based approaches, are introduced. These agents utilize CNNs and LSTMs as parametric function approximators to map historical price observations to market positions.Backtesting on front-month natural gas futures demonstrates that DRL models increase the Sharpe ratio by $83\%$ compared to the buy-and-hold baseline. Additionally, the risk profile of the agents can be customized through a hyperparameter that regulates risk sensitivity in the reward function during the optimization process. The actor-based models outperform the actor-critic-based models, while the CNN-based models show a slight performance advantage over the LSTM-based models.