Variable Coded Batch Matrix Multiplication

TL;DR

This paper introduces Variable Coded Distributed Batch Matrix Multiplication (VCDBMM) and Flexible Cross-Subspace Alignments (FCSA) codes, which leverage matrix reuse across batch jobs to improve straggler resilience and decoding speed.

Contribution

The paper proposes FCSA codes for VCDBMM, enabling efficient matrix reuse, and provides a theoretical and empirical analysis of their near-optimal performance.

Findings

FCSA codes are within a factor of 2 of optimal for straggler resilience.

Simulations show FCSA codes can achieve an optimality gap as low as 1.7.

The approach improves upon existing codes by exploiting matrix redundancy across batches.

Abstract

A majority of coded matrix-matrix computation literature has broadly focused in two directions: matrix partitioning for computing a single computation task and batch processing of multiple distinct computation tasks. While these works provide codes with good straggler resilience and fast decoding for their problem spaces, these codes would not be able to take advantage of the natural redundancy of re-using matrices across batch jobs. In this paper, we introduce the Variable Coded Distributed Batch Matrix Multiplication (VCDBMM) problem which tasks a distributed system to perform batch matrix multiplication where matrices are not necessarily distinct among batch jobs. Inspired in part by Cross-Subspace Alignment codes, we develop Flexible Cross-Subspace Alignments (FCSA) codes that are flexible enough to utilize this redundancy. We provide a full characterization of FCSA codes which allow for a wide variety of system complexities including good straggler resilience and fast decoding. We theoretically demonstrate that, under certain practical conditions, FCSA codes are within a factor of 2 of the optimal solution when it comes to straggler resilience. Furthermore, our simulations demonstrate that our codes can achieve even better optimality gaps in practice, even going as low as 1.7.