Approximate Weighted $CR$ Coded Matrix Multiplication

TL;DR

This paper introduces a novel approximate weighted CR-coded matrix multiplication scheme designed to improve the efficiency of distributed matrix multiplication, especially in environments with straggler workers, addressing a key computational bottleneck in machine learning.

Contribution

The paper proposes a new weighted CR-coded matrix multiplication method that enhances performance in distributed settings with stragglers, combining approximation and coding strategies.

Findings

Improved performance in distributed matrix multiplication

Effective handling of straggler workers

Enhanced efficiency over existing methods

Abstract

One of the most common, but at the same time expensive operations in linear algebra, is multiplying two matrices $A$ and $B$. With the rapid development of machine learning and increases in data volume, performing fast matrix intensive multiplications has become a major hurdle. Two different approaches to overcoming this issue are, 1) to approximate the product; and 2) to perform the multiplication distributively. A \textit{$CR$-multiplication} is an approximation where columns and rows of $A$ and $B$ are respectively sampled with replacement. In the distributed setting, multiple workers perform matrix multiplication subtasks in parallel. Some of the workers may be stragglers, meaning they do not complete their task in time. We present a novel \textit{approximate weighted $CR$ coded matrix multiplication} scheme, that achieves improved performance for distributed matrix multiplication.