Multipass Linear Sketches for Geometric LP-Type Problems

TL;DR

This paper introduces multipass linear sketching algorithms for LP-type problems in streaming and distributed models, achieving high accuracy with low-dimensional data and exponential improvements over previous methods.

Contribution

It presents a novel multipass linear sketching framework for LP-type problems, providing algorithms with low space complexity and significant improvements in approximation accuracy.

Findings

Achieves polynomial space complexity in data dimension d

Provides exponential improvement in 1/ε approximation

Establishes lower bounds motivating multi-pass algorithms

Abstract

LP-type problems such as the Minimum Enclosing Ball (MEB), Linear Support Vector Machine (SVM), Linear Programming (LP), and Semidefinite Programming (SDP) are fundamental combinatorial optimization problems, with many important applications in machine learning applications such as classification, bioinformatics, and noisy learning. We study LP-type problems in several streaming and distributed big data models, giving $\varepsilon$-approximation linear sketching algorithms with a focus on the high accuracy regime with low dimensionality $d$, that is, when ${d < (1/\varepsilon)^{0.999}}$. Our main result is an $O(ds)$ pass algorithm with $O(s( \sqrt{d}/\varepsilon)^{3d/s}) \cdot \mathrm{poly}(d, \log (1/\varepsilon))$ space complexity in words, for any parameter $s \in [1, d \log (1/\varepsilon)]$, to solve $\varepsilon$-approximate LP-type problems of $O(d)$ combinatorial and VC dimension. Notably, by taking $s = d \log (1/\varepsilon)$, we achieve space complexity polynomial in $d$ and polylogarithmic in $1/\varepsilon$, presenting exponential improvements in $1/\varepsilon$ over current algorithms. We complement our results by showing lower bounds of $(1/\varepsilon)^{\Omega(d)}$ for any $1$-pass algorithm solving the $(1 + \varepsilon)$-approximation MEB and linear SVM problems, further motivating our multi-pass approach.