A Memory Hierarchical Layer Assigning and Prefetching Technique to Overcome the Memory Performance/Energy Bottleneck

TL;DR

This paper introduces a formalized memory hierarchy and prefetching technique that optimizes performance and energy efficiency by systematically exploring trade-offs, demonstrated on real applications with significant improvements.

Contribution

It presents a novel formalized approach for memory layer assignment and prefetching that considers data reuse and application-specific factors, filling a gap in systematic trade-off analysis.

Findings

Reduced execution time by up to 60%

Decreased energy consumption by up to 70%

Validated on nine real-world applications

Abstract

The memory subsystem has always been a bottleneck in performance as well as significant power contributor in memory intensive applications. Many researchers have presented multi-layered memory hierarchies as a means to design energy and performance efficient systems. However, most of the previous work do not explore trade-offs systematically. We fill this gap by proposing a formalized technique that takes into consideration data reuse, limited lifetime of the arrays of an application and application specific prefetching opportunities, and performs a thorough trade-off exploration for different memory layer sizes. This technique has been implemented on a prototype tool, which was tested successfully using nine real-life applications of industrial relevance. Following this approach we have able to reduce execution time up to 60%, and energy consumption up to 70%.