Five-Minute Rule 40 Years Later: A First-Principles Revisit for Modern Memory Hierarchy

TL;DR

This paper revisits the five-minute rule for memory hierarchy, integrating modern hardware costs and performance models to provide actionable guidance for AI platforms, especially with GPU and SSD advancements.

Contribution

It extends the classical heuristic into a first-principles, workload-aware framework tailored for modern AI hardware and introduces MQSim-Next, a new SSD simulation tool.

Findings

DRAM-SSD caching threshold drops from minutes to seconds for AI workloads.

NAND flash memory acts as an active data tier in modern systems.

The framework enables new hardware-software co-design research avenues.

Abstract

In 1987, Jim Gray and Gianfranco Putzolu introduced the five-minute rule, a simple, storage-memory-economics-based heuristic for deciding when data should live in DRAM rather than on storage. Subsequent revisits to the rule largely retained that economics-only view, leaving host costs, feasibility limits, and workload behavior out of scope. This paper revisits the rule from first principles, integrating host costs, DRAM bandwidth/capacity, and physics-grounded models of SSD performance and cost, and then embedding these elements in a constraint- and workload-aware framework that yields actionable provisioning guidance. We show that, for modern AI platforms, especially GPU-centric hosts paired with ultra-high-IOPS SSDs engineered for fine-grained random access, the DRAM$\leftrightarrow$flash caching threshold collapses from minutes to a few seconds. This shift reframes NAND flash memory as an \emph{active data tier} and exposes a broad research space across the hardware-software stack. We further introduce MQSim-Next, a calibrated SSD simulator that supports validation and sensitivity analysis and facilitates future architectural and system research. Finally, we present two concrete case studies that showcase the software system design space opened by such memory hierarchy paradigm shift. Overall, we turn a classical heuristic into an actionable, feasibility-aware analysis and provisioning framework and set the stage for further research on AI-era memory hierarchy.