Sketching with Kerdock's crayons: Fast sparsifying transforms for arbitrary linear maps

TL;DR

This paper introduces a randomized, fast sparsifying transform for arbitrary matrices, enabling efficient approximation of sparse matrix-vector products using Kerdock-based representations.

Contribution

It develops a novel randomized approach utilizing Kerdock sets to efficiently approximate sparse matrix-vector products for unstructured matrices.

Findings

Preprocessing of matrix A takes O(n^3 log n) operations.

The transform computes sparse approximations in O(sn + ε^{-2}‖A‖_{2→∞}^2 n log n) time.

Numerical results support the practical feasibility of the proposed method.

Abstract

Given an arbitrary matrix $A\in\mathbb{R}^{n\times n}$, we consider the fundamental problem of computing $Ax$ for any $x\in\mathbb{R}^n$ such that $Ax$ is $s$-sparse. While fast algorithms exist for particular choices of $A$, such as the discrete Fourier transform, there is currently no $o(n^2)$ algorithm that treats the unstructured case. In this paper, we devise a randomized approach to tackle the unstructured case. Our method relies on a representation of $A$ in terms of certain real-valued mutually unbiased bases derived from Kerdock sets. In the preprocessing phase of our algorithm, we compute this representation of $A$ in $O(n^3\log n)$ operations. Next, given any unit vector $x\in\mathbb{R}^n$ such that $Ax$ is $s$-sparse, our randomized fast transform uses this representation of $A$ to compute the entrywise $\epsilon$-hard threshold of $Ax$ with high probability in only $O(sn + \epsilon^{-2}\|A\|_{2\to\infty}^2n\log n)$ operations. In addition to a performance guarantee, we provide numerical results that demonstrate the plausibility of real-world implementation of our algorithm.