Distributed Structured Matrix Multiplication

TL;DR

This paper introduces novel encoding schemes for distributed source compression to efficiently compute matrix products, showing significant rate savings especially with correlated sources, advancing the field of distributed computation.

Contribution

It proposes nonlinear mappings combined with structured linear encoding for distributed matrix computations, demonstrating potential for unbounded compression gains with correlated sources.

Findings

Achievable sum rate improvements over existing methods.

Significant compression savings with correlated sources.

Numerical simulations confirm theoretical gains.

Abstract

We devise achievable encoding schemes for distributed source compression for computing inner products, symmetric matrix products, and more generally, square matrix products, which are a class of nonlinear transformations. To that end, our approach relies on devising nonlinear mappings of distributed sources, which are then followed by the structured linear encoding scheme, introduced by K\"orner and Marton. For different computation scenarios, we contrast our findings on the achievable sum rate with the state of the art to demonstrate the possible savings in compression rate. When the sources have special correlation structures, it is possible to achieve unbounded gains, as demonstrated by the analysis and numerical simulations.