Variable Coded Batch Matrix Multiplication
Lev Tauz, Lara Dolecek

TL;DR
This paper introduces Variable Coded Distributed Batch Matrix Multiplication (VCDBMM) and Flexible Cross-Subspace Alignments (FCSA) codes, which leverage matrix reuse across batch jobs to improve straggler resilience and decoding speed.
Contribution
The paper proposes FCSA codes for VCDBMM, enabling efficient matrix reuse, and provides a theoretical and empirical analysis of their near-optimal performance.
Findings
FCSA codes are within a factor of 2 of optimal for straggler resilience.
Simulations show FCSA codes can achieve an optimality gap as low as 1.7.
The approach improves upon existing codes by exploiting matrix redundancy across batches.
Abstract
A majority of coded matrix-matrix computation literature has broadly focused in two directions: matrix partitioning for computing a single computation task and batch processing of multiple distinct computation tasks. While these works provide codes with good straggler resilience and fast decoding for their problem spaces, these codes would not be able to take advantage of the natural redundancy of re-using matrices across batch jobs. In this paper, we introduce the Variable Coded Distributed Batch Matrix Multiplication (VCDBMM) problem which tasks a distributed system to perform batch matrix multiplication where matrices are not necessarily distinct among batch jobs. Inspired in part by Cross-Subspace Alignment codes, we develop Flexible Cross-Subspace Alignments (FCSA) codes that are flexible enough to utilize this redundancy. We provide a full characterization of FCSA codes which…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsStochastic Gradient Optimization Techniques · Error Correcting Code Techniques · Parallel Computing and Optimization Techniques
