Improved Practical Matrix Sketching with Guarantees
Amey Desai, Mina Ghashami, Jeff M. Phillips

TL;DR
This paper compares and improves matrix sketching methods, balancing error, size, and computational efficiency, and introduces practical modifications to existing guarantees for better real-world performance.
Contribution
It categorizes known matrix sketching methods, proposes modifications for practical improvements while maintaining guarantees, and provides a reproducible testbed for evaluation.
Findings
iSVD heuristic outperforms known methods in size/error trade-off
Modified FrequentDirections matches iSVD performance with guarantees
Hashing and sampling methods excel in time/error trade-off
Abstract
Matrices have become essential data representations for many large-scale problems in data analytics, and hence matrix sketching is a critical task. Although much research has focused on improving the error/size tradeoff under various sketching paradigms, the many forms of error bounds make these approaches hard to compare in theory and in practice. This paper attempts to categorize and compare most known methods under row-wise streaming updates with provable guarantees, and then to tweak some of these methods to gain practical improvements while retaining guarantees. For instance, we observe that a simple heuristic iSVD, with no guarantees, tends to outperform all known approaches in terms of size/error trade-off. We modify the best performing method with guarantees FrequentDirections under the size/error trade-off to match the performance of iSVD and retain its guarantees. We also…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAlgorithms and Data Compression · Stochastic Gradient Optimization Techniques · Advanced Image and Video Retrieval Techniques
