Oblivious sketching for logistic regression

TL;DR

This paper introduces the first data oblivious sketching method for logistic regression that enables one-pass streaming solutions with strong approximation guarantees, offering a practical and efficient approach for large-scale data analysis.

Contribution

The paper presents a novel data oblivious sketch for logistic regression that operates efficiently in streaming settings and provides provable approximation guarantees.

Findings

Sketch reduces data size from n to poly(μ d log n) points

Achieves O(log n)-approximation for logistic regression

Practical, fast, and easy-to-implement method

Abstract

What guarantees are possible for solving logistic regression in one pass over a data stream? To answer this question, we present the first data oblivious sketch for logistic regression. Our sketch can be computed in input sparsity time over a turnstile data stream and reduces the size of a $d$-dimensional data set from $n$ to only $\operatorname{poly}(\mu d\log n)$ weighted points, where $\mu$ is a useful parameter which captures the complexity of compressing the data. Solving (weighted) logistic regression on the sketch gives an $O(\log n)$-approximation to the original problem on the full data set. We also show how to obtain an $O(1)$-approximation with slight modifications. Our sketches are fast, simple, easy to implement, and our experiments demonstrate their practicality.