Fast and Guaranteed Tensor Decomposition via Sketching
Yining Wang, Hsiao-Yu Tung, Alexander Smola, Animashree Anandkumar

TL;DR
This paper introduces fast, randomized tensor CP decomposition algorithms using sketching techniques, enabling efficient computation without explicitly forming large tensors, applicable to both sparse and dense data.
Contribution
The paper presents novel tensor-specific sketching methods, including FFT-based contractions and colliding hashes, to accelerate tensor decomposition algorithms.
Findings
Achieves faster tensor CP decomposition without sacrificing accuracy
Applicable to both sparse and dense tensors with consistent quality
Demonstrates competitive results in topic modeling applications
Abstract
Tensor CANDECOMP/PARAFAC (CP) decomposition has wide applications in statistical learning of latent variable models and in data mining. In this paper, we propose fast and randomized tensor CP decomposition algorithms based on sketching. We build on the idea of count sketches, but introduce many novel ideas which are unique to tensors. We develop novel methods for randomized computation of tensor contractions via FFTs, without explicitly forming the tensors. Such tensor contractions are encountered in decomposition methods such as tensor power iterations and alternating least squares. We also design novel colliding hashes for symmetric tensors to further save time in computing the sketches. We then combine these sketching ideas with existing whitening and tensor power iterative techniques to obtain the fastest algorithm on both sparse and dense tensors. The quality of approximation under…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTensor decomposition and applications · Machine Learning and Algorithms · Stochastic Gradient Optimization Techniques
