A Coreset Learning Reality Check

TL;DR

This paper critically evaluates various subsampling methods for logistic regression, revealing that many do not outperform simple uniform sampling and highlighting inconsistencies in their effectiveness.

Contribution

It provides the first comprehensive comparison of coreset and optimal subsampling methods for logistic regression, exposing their limitations and inconsistencies.

Findings

Many advanced subsampling methods do not outperform uniform sampling.

Inconsistencies exist in the effectiveness of different subsampling techniques.

Some methods underperform despite theoretical guarantees.

Abstract

Subsampling algorithms are a natural approach to reduce data size before fitting models on massive datasets. In recent years, several works have proposed methods for subsampling rows from a data matrix while maintaining relevant information for classification. While these works are supported by theory and limited experiments, to date there has not been a comprehensive evaluation of these methods. In our work, we directly compare multiple methods for logistic regression drawn from the coreset and optimal subsampling literature and discover inconsistencies in their effectiveness. In many cases, methods do not outperform simple uniform subsampling.