Lightweight ML-based Runtime Prefetcher Selection on Many-core Platforms

TL;DR

This paper introduces a lightweight supervised learning approach for runtime prefetcher selection in many-core systems, significantly improving performance by managing composite prefetchers more effectively.

Contribution

It presents a novel phase-based training methodology for a lightweight model to optimize prefetcher selection, addressing scalability issues of prior heuristics and deep reinforcement learning methods.

Findings

Up to 25% performance improvement on a state-of-the-art system

Average 2.7% performance gain over default configuration

Effective management of composite prefetchers in many-core environments

Abstract

Modern computer designs support composite prefetching, where multiple individual prefetcher components are used to target different memory access patterns. However, multiple prefetchers competing for resources can drastically hurt performance, especially in many-core systems where cache and other resources are shared and very limited. Prior work has proposed mitigating this issue by selectively enabling and disabling prefetcher components during runtime. Traditional approaches proposed heuristics that are hard to scale with increasing core and prefetcher component counts. More recently, deep reinforcement learning was proposed. However, it is too expensive to deploy in real-world many-core systems. In this work, we propose a new phase-based methodology for training a lightweight supervised learning model to manage composite prefetchers at runtime. Our approach improves the performance of a state-of-the-art many-core system by up to 25% and by 2.7% on average over its default prefetcher configuration.