Matrix Multiplication in the MPC Model

TL;DR

This paper develops efficient algorithms for matrix multiplication within the MPC model, optimizing for various matrix sizes and sparsity, and analyzing the trade-offs between rounds, processor count, and memory constraints.

Contribution

It introduces new algorithms for matrix multiplication in the MPC model under different processor and memory constraints, including sparse matrices, with proven round complexity bounds.

Findings

Matrix multiplication of rectangular matrices achieved in sublinear rounds.

Algorithms tailored for sparse matrices with efficient round complexity.

Trade-offs between processor count, memory, and rounds are characterized.

Abstract

In this paper, we present algorithms to solve matrix multiplication problems in the MPC model. In particular, we consider the problem under various processor/memory constraints in the MPC model and prove the following results. 1. Multiplication of two rectangular matrices of size $d \times n$ and $n \times d$ ( where $d \leq n$) respectively can be done in, i) $O(\sqrt{d} + \log_d n)$ rounds with $n$ processors and $\Theta(d)$ memory per processor ii) $O(\frac{d}{\sqrt{n}})$ rounds with $d$ processors and $\Theta(n)$ memory per processor. 2. Multiplication of two rectangular matrices of size $n \times d$ and $d \times n$ (where $d \leq n$) respectively, with $n$ processors of $\Theta(n)$ memory per processor, can be done in $O(\frac{d}{\sqrt{n}})$ rounds. 3.The multiplication of two $d$-sparse matrices (matrices that contain at most $d$-nonzero elements in each row and in each column) with $n$ processors and $\Theta(d)$ memory per processor can be done in $O(d^{0.9})$ rounds.