Distributed Matrix Tiling Using A Hypergraph Labeling Formulation

TL;DR

This paper models the problem of partitioning large matrices for distributed linear algebra as a hypergraph labeling problem, providing theoretical insights and a practical greedy algorithm with experimental validation.

Contribution

It introduces a hypergraph labeling formulation for matrix tiling, proves hardness results, and develops a greedy algorithm with experimental evaluation.

Findings

Proved hardness of the matrix tiling hypergraph labeling problem.

Provided a theoretical complexity characterization on random instances.

Demonstrated the effectiveness of the greedy algorithm through experiments.

Abstract

Partitioning large matrices is an important problem in distributed linear algebra computing (used in ML among others). Briefly, our goal is to perform a sequence of matrix algebra operations in a distributed manner (whenever possible) on these large matrices. However, not all partitioning schemes work well with different matrix algebra operations and their implementations (algorithms). This is a type of data tiling problem. In this work we consider a theoretical model for a version of the matrix tiling problem in the setting of hypergraph labeling. We prove some hardness results and give a theoretical characterization of its complexity on random instances. Additionally we develop a greedy algorithm and experimentally show its efficacy.