Tight Memory-Independent Parallel Matrix Multiplication Communication Lower Bounds

TL;DR

This paper establishes tight, memory-independent communication lower bounds for parallel matrix multiplication, improving constants over previous work and aiding in the development of more efficient algorithms.

Contribution

It provides the first tight, memory-independent communication lower bounds with precise constants for parallel matrix multiplication, considering different matrix aspect ratios.

Findings

Constants improve on previous bounds in all cases

Bounds are memory-independent and tight

Results guide more efficient algorithm design

Abstract

Communication lower bounds have long been established for matrix multiplication algorithms. However, most methods of asymptotic analysis have either ignored the constant factors or not obtained the tightest possible values. Recent work has demonstrated that more careful analysis improves the best known constants for some classical matrix multiplication lower bounds and helps to identify more efficient algorithms that match the leading-order terms in the lower bounds exactly and improve practical performance. The main result of this work is the establishment of memory-independent communication lower bounds with tight constants for parallel matrix multiplication. Our constants improve on previous work in each of three cases that depend on the relative sizes of the aspect ratios of the matrices.