Fine-tuning Timeseries Predictors Using Reinforcement Learning

TL;DR

This paper explores reinforcement learning algorithms for fine-tuning financial time series predictors, demonstrating performance improvements and transfer learning benefits through empirical evaluation.

Contribution

It introduces a clear implementation plan for backpropagating reinforcement learning loss to supervised models and compares pre- and post-fine-tuning performance.

Findings

Performance increases after fine-tuning

Transfer learning properties observed in models

Empirical results support practical implementation

Abstract

This chapter presents three major reinforcement learning algorithms used for fine-tuning financial forecasters. We propose a clear implementation plan for backpropagating the loss of a reinforcement learning task to a model trained using supervised learning, and compare the performance before and after the fine-tuning. We find an increase in performance after fine-tuning, and transfer learning properties to the models, indicating the benefits of fine-tuning. We also highlight the tuning process and empirical results for future implementation by practitioners.