
TL;DR
This paper introduces a simple, FFT-based algorithm for approximating and exactly computing matrix products efficiently, especially when the product is sparse, using hashing, error-correcting codes, and probabilistic techniques.
Contribution
The paper presents a novel compressed matrix multiplication algorithm that leverages polynomial hashing, FFT, and error-correcting codes for efficient approximation and exact computation.
Findings
Efficient approximation of matrix products with controlled variance.
Near-linear time exact computation for sparse matrices with few nonzero entries.
Use of error-correcting codes to recover significant entries in matrix products.
Abstract
Motivated by the problems of computing sample covariance matrices, and of transforming a collection of vectors to a basis where they are sparse, we present a simple algorithm that computes an approximation of the product of two n-by-n real matrices A and B. Let ||AB||_F denote the Frobenius norm of AB, and b be a parameter determining the time/accuracy trade-off. Given 2-wise independent hash functions $_1,h_2: [n] -> [b], and s_1,s_2: [n] -> {-1,+1} the algorithm works by first "compressing" the matrix product into the polynomial p(x) = sum_{k=1}^n (sum_{i=1}^n A_{ik} s_1(i) x^{h_1(i)}) (sum_{j=1}^n B_{kj} s_2(j) x^{h_2(j)}) Using FFT for polynomial multiplication, we can compute c_0,...,c_{b-1} such that sum_i c_i x^i = (p(x) mod x^b) + (p(x) div x^b) in time \~O(n^2+ n b). An unbiased estimator of (AB)_{ij} with variance at most ||AB||_F^2 / b can then be computed as: C_{ij} =…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsError Correcting Code Techniques · Algorithms and Data Compression · Complexity and Algorithms in Graphs
