Branch Prediction as a Reinforcement Learning Problem: Why, How and Case Studies

TL;DR

This paper proposes viewing branch prediction as a reinforcement learning problem to enable systematic design and analysis, demonstrating how existing predictors fit this framework and exploring new RL-based variants.

Contribution

It introduces a reinforcement learning formulation for branch prediction, unifies existing predictors under this framework, and presents novel RL-based predictor variants.

Findings

Existing predictors can be expressed as RL models

RL-based variants outperform traditional predictors in certain scenarios

Framework facilitates systematic exploration of new BP designs

Abstract

Recent years have seen stagnating improvements to branch predictor (BP) efficacy and a dearth of fresh ideas in branch predictor design, calling for fresh thinking in this area. This paper argues that looking at BP from the viewpoint of Reinforcement Learning (RL) facilitates systematic reasoning about, and exploration of, BP designs. We describe how to apply the RL formulation to branch predictors, show that existing predictors can be succinctly expressed in this formulation, and study two RL-based variants of conventional BPs.