Hierarchical Reinforcement Learning for Temporal Pattern Prediction

TL;DR

This paper demonstrates that feudal hierarchical reinforcement learning enhances training speed, stability, and accuracy in temporal sequence prediction tasks across finance and autonomous driving domains by leveraging multi-resolution abstractions.

Contribution

It introduces a multi-resolution hierarchical RL framework that improves prediction performance and training efficiency in temporal sequence tasks.

Findings

Feudal RL significantly improves training speed.

Multi-resolution structure enhances prediction accuracy.

Applicable to finance and autonomous driving domains.

Abstract

In this work, we explore the use of hierarchical reinforcement learning (HRL) for the task of temporal sequence prediction. Using a combination of deep learning and HRL, we develop a stock agent to predict temporal price sequences from historical stock price data and a vehicle agent to predict steering angles from first person, dash cam images. Our results in both domains indicate that a type of HRL, called feudal reinforcement learning, provides significant improvements to training speed and stability and prediction accuracy over standard RL. A key component to this success is the multi-resolution structure that introduces both temporal and spatial abstraction into the network hierarchy.