The Recurrent Reinforcement Learning Crypto Agent

TL;DR

This paper presents a novel online transfer learning approach using echo state networks and reinforcement learning for intraday Bitcoin trading, achieving significant returns and risk-adjusted performance over five years.

Contribution

It introduces a new crypto trading agent combining echo state networks with reinforcement learning and online transfer learning, demonstrating effective market prediction and profit generation.

Findings

Total return of 350% over five years

71% of profit from funding strategies

Annualized information ratio of 1.46

Abstract

We demonstrate a novel application of online transfer learning for a digital assets trading agent. This agent uses a powerful feature space representation in the form of an echo state network, the output of which is made available to a direct, recurrent reinforcement learning agent. The agent learns to trade the XBTUSD (Bitcoin versus US Dollars) perpetual swap derivatives contract on BitMEX on an intraday basis. By learning from the multiple sources of impact on the quadratic risk-adjusted utility that it seeks to maximise, the agent avoids excessive over-trading, captures a funding profit, and can predict the market's direction. Overall, our crypto agent realises a total return of 350\%, net of transaction costs, over roughly five years, 71\% of which is down to funding profit. The annualised information ratio that it achieves is 1.46.