Improved Parallel Cache-Oblivious Algorithms for Dynamic Programming and Linear Algebra

TL;DR

This paper introduces improved cache-oblivious algorithms for dynamic programming and linear algebra tailored for non-volatile memory, reducing write costs and enhancing parallelism by leveraging a novel $k$-d grid abstraction.

Contribution

The paper presents a new framework using $k$-d grids to develop cache-oblivious algorithms that minimize writes in asymmetric memory settings, applicable to many DP and linear algebra problems.

Findings

New algorithms reduce cache complexity in asymmetric memory environments.

The $k$-d grid abstraction enhances understanding of problem complexity.

Algorithms outperform existing methods even without considering write asymmetry.

Abstract

Emerging non-volatile main memory (NVRAM) technologies provide byte-addressability, low idle power, and improved memory-density, and are likely to be a key component in the future memory hierarchy. However, a critical challenge in achieving high performance is in accounting for the asymmetry that NVRAM writes can be significantly more expensive than NVRAM reads. In this paper, we consider a large class of cache-oblivious algorithms for dynamic programming (DP) and linear algebra, and try to reduce the writes in the asymmetric setting while maintaining high parallelism. To achieve that, our key approach is to show the correspondence between these problems and an abstraction for their computation, which is referred to as the $k$-d grids. Then by showing lower bound and new algorithms for computing $k$-d grids, we show a list of improved cache-oblivious algorithms of many DP recurrences and in linear algebra in the asymmetric setting, both sequentially and in parallel. Surprisingly, even without considering the read-write asymmetry (i.e., setting the write cost to be the same as the read cost in the algorithms), the new algorithms improve the existing cache complexity of many problems. We believe the reason is that the extra level of abstraction of $k$-d grids helps us to better understand the complexity and difficulties of these problems. We believe that the novelty of our framework is of interests and leads to many new questions for future work.