On-Chip Mechanisms to Reduce Effective Memory Access Latency

TL;DR

This paper introduces on-chip hardware mechanisms that classify cache misses and accelerate their handling, significantly reducing memory access latency and improving performance in multi-core systems.

Contribution

It proposes novel hardware techniques to classify cache misses and migrate dependence chains, accelerating memory access and reducing latency.

Findings

62% performance increase on high memory workloads

19% reduction in effective memory access latency

Effective in both single-core and multi-core systems

Abstract

This dissertation develops hardware that automatically reduces the effective latency of accessing memory in both single-core and multi-core systems. To accomplish this, the dissertation shows that all last level cache misses can be separated into two categories: dependent cache misses and independent cache misses. Independent cache misses have all of the source data that is required to generate the address of the memory access available on-chip, while dependent cache misses depend on data that is located off-chip. This dissertation proposes that dependent cache misses are accelerated by migrating the dependence chain that generates the address of the memory access to the memory controller for execution. Independent cache misses are accelerated using a new mode for runahead execution that only executes filtered dependence chains. With these mechanisms, this dissertation demonstrates a 62% increase in performance and a 19% decrease in effective memory access latency for a quad-core processor on a set of high memory intensity workloads.