Structured Codes for Distributed Matrix Multiplication

TL;DR

This paper establishes the fundamental limits and optimal coding schemes for distributed computation of bilinear functions, like matrix products, over finite fields with correlated sources, improving over traditional methods.

Contribution

It introduces tight bounds and achievable schemes for distributed bilinear function computation, leveraging structured codes and source correlation insights.

Findings

Derived tight bounds on sum rate for bilinear function computation.

Achieved unbounded compression gains over Slepian-Wolf coding.

Characterized fundamental limits for distributed matrix multiplication over finite fields.

Abstract

Our work addresses the well-known open problem of distributed computing of bilinear functions of two correlated sources ${\bf A}$ and ${\bf B}$. In a setting with two nodes, with the first node having access to ${\bf A}$ and the second to ${\bf B}$, we establish bounds on the optimal sum rate that allows a receiver to compute an important class of non-linear functions, and in particular bilinear functions, including dot products $\langle {\bf A},{\bf B}\rangle$, and general matrix products ${\bf A}^{\intercal}{\bf B}$ over finite fields. The bounds are tight for large field sizes, for which case we can derive the exact fundamental performance limits for all problem dimensions and a large class of sources. Our achievability scheme involves the design of non-linear transformations of ${\bf A}$ and ${\bf B}$, carefully calibrated to work synergistically with the structured linear encoding scheme by K\"orner and Marton. The subsequent converses derived here, calibrate the Han-Kobayashi approach and the strong converse of Ahlswede-G\'acs-K\"orner to yield relatively tight converses on the sum rate. We exhibit unbounded compression gains over Slepian-Wolf coding, depending on the source correlations. In the end, this work characterizes the fundamental limits of distributed computing for a crucial class of functions, while succinctly capturing the inherent computation structures and source correlations.