Matrix Multiplication and Binary Space Partitioning Trees : An Exploration

TL;DR

This paper investigates a dual tree algorithm for matrix multiplication that is effective under specific clustering conditions of normalized vectors but faces exponential challenges with uniform distributions in high dimensions.

Contribution

The paper introduces a dual tree algorithm for matrix multiplication that exploits clustering of normalized vectors to improve efficiency under certain conditions.

Findings

Effective when vectors cluster tightly on the unit sphere.

Performance degrades exponentially with increasing dimension for uniform distributions.

Requires significant work to be practical in high-dimensional uniform cases.

Abstract

Herein we explore a dual tree algorithm for matrix multiplication of $A\in \mathbb{R}^{M\times D}$ and $B\in\mathbb{R}^{D\times N}$, very narrowly effective if the normalized rows of $A$ and columns of $B$, treated as vectors in $\mathbb{R}^{D}$, fall into clusters of order proportionate to $\Omega(D^{\tau})$ with radii less than $\arcsin(\epsilon/\sqrt{2})$ on the surface of the unit $D$-ball. The algorithm leverages a pruning rule necessary to guarantee $\epsilon$ precision proportionate to vector magnitude products in the resultant matrix. \textit{ Unfortunately, if the rows and columns are uniformly distributed on the surface of the unit $D$-ball, then the expected points per required cluster approaches zero exponentially fast in $D$; thus, the approach requires a great deal of work to pass muster.}