A Short Note on Batch-efficient Divide-and-Conquer Algorithm for EigenDecomposition

TL;DR

This paper introduces a batch-efficient Divide-and-Conquer eigen-decomposition algorithm optimized for matrices smaller than 64, significantly speeding up computations in deep learning applications.

Contribution

It extends previous QR-based methods to larger matrices using Divide-and-Conquer, improving efficiency for mini-batch eigen-decomposition tasks.

Findings

Faster than Pytorch SVD for matrices under 64 dimensions

Effective for mini-batch processing in neural networks

Improves computational bottleneck in eigen-decomposition

Abstract

EigenDecomposition (ED) is at the heart of many computer vision algorithms and applications. One crucial bottleneck limiting its usage is the expensive computation cost, particularly for a mini-batch of matrices in deep neural networks. Our previous work proposed a dedicated QR-based ED algorithm for batched small matrices (dim${<}32$). This short paper targets the limitation and proposes a batch-efficient Divide-and-Conquer based ED algorithm for larger matrices. The numerical test shows that for a mini-batch of matrices whose dimensions are smaller than $64$, our method can be much faster than the Pytorch SVD function.