On Coresets for Logistic Regression

TL;DR

This paper investigates the limits of coreset construction for logistic regression, introduces a complexity measure to identify tractable cases, and proposes a novel sampling method that produces sublinear coresets with theoretical guarantees, validated on real data.

Contribution

The paper introduces a complexity measure for logistic regression data sets and a new sensitivity sampling scheme that yields the first provably sublinear coresets under certain conditions.

Findings

No strongly sublinear coresets exist in the worst case.

The proposed method produces sublinear coresets for data with bounded complexity.

Experimental results show improved performance over uniform sampling and existing methods.

Abstract

Coresets are one of the central methods to facilitate the analysis of large data sets. We continue a recent line of research applying the theory of coresets to logistic regression. First, we show a negative result, namely, that no strongly sublinear sized coresets exist for logistic regression. To deal with intractable worst-case instances we introduce a complexity measure $\mu(X)$, which quantifies the hardness of compressing a data set for logistic regression. $\mu(X)$ has an intuitive statistical interpretation that may be of independent interest. For data sets with bounded $\mu(X)$-complexity, we show that a novel sensitivity sampling scheme produces the first provably sublinear $(1\pm\varepsilon)$-coreset. We illustrate the performance of our method by comparing to uniform sampling as well as to state of the art methods in the area. The experiments are conducted on real world benchmark data for logistic regression.