Minimizing Communication for Parallel Symmetric Tensor Times Same Vector Computation

TL;DR

This paper establishes fundamental lower bounds on communication costs for parallel symmetric tensor-vector multiplication and presents an optimal algorithm matching these bounds, improving efficiency in tensor eigenpair computations.

Contribution

It introduces tight communication lower bounds for symmetric tensor-vector multiplication and proposes an optimal algorithm that achieves these bounds.

Findings

Derived tight communication lower bounds for the computation.

Presented an optimal algorithm matching the lower bounds.

Extended geometric inequalities to 3D symmetric tensor computations.

Abstract

In this article, we focus on the parallel communication cost of multiplying the same vector along two modes of a $3$-dimensional symmetric tensor. This is a key computation in the higher-order power method for determining eigenpairs of a $3$-dimensional symmetric tensor and in gradient-based methods for computing a symmetric CP decomposition. We establish communication lower bounds that determine how much data movement is required to perform the specified computation in parallel. The core idea of the proof relies on extending a key geometric inequality for $3$-dimensional symmetric computations. We demonstrate that the communication lower bounds are tight by presenting an optimal algorithm where the data distribution is a natural extension of the triangle block partition scheme for symmetric matrices to 3-dimensional symmetric tensors.