On Symmetric Rectilinear Matrix Partitioning

TL;DR

This paper studies symmetric rectilinear partitioning of square matrices, proves its NP-hardness, proposes heuristics, and demonstrates their efficiency and effectiveness on large sparse matrices, achieving near-perfect load balance.

Contribution

It introduces the first heuristics for symmetric rectilinear matrix partitioning, analyzes their complexity, and improves their practicality with sparsification and efficient data structures.

Findings

Heuristics solve large matrices in under 3 seconds.

Solutions achieve load imbalance no worse than 1%.

NP-hardness of the problem is established.

Abstract

Even distribution of irregular workload to processing units is crucial for efficient parallelization in many applications. In this work, we are concerned with a spatial partitioning called rectilinear partitioning (also known as generalized block distribution) of sparse matrices. More specifically, in this work, we address the problem of symmetric rectilinear partitioning of a square matrix. By symmetric, we mean the rows and columns of the matrix are identically partitioned yielding a tiling where the diagonal tiles (blocks) will be squares. We first show that the optimal solution to this problem is NP-hard, and we propose four heuristics to solve two different variants of this problem. We present a thorough analysis of the computational complexities of those proposed heuristics. To make the proposed techniques more applicable in real life application scenarios, we further reduce their computational complexities by utilizing effective sparsification strategies together with an efficient sparse prefix-sum data structure. We experimentally show the proposed algorithms are efficient and effective on more than six hundred test matrices. With sparsification, our methods take less than 3 seconds in the Twitter graph on a modern 24 core system and output a solution whose load imbalance is no worse than 1%.