Screening Data Points in Empirical Risk Minimization via Ellipsoidal Regions and Safe Loss Functions

TL;DR

This paper introduces a method for efficiently screening data points in empirical risk minimization using ellipsoidal regions and safe loss functions, enabling dataset compression and computational savings.

Contribution

It proposes novel screening tests and loss functions that induce dual sparsity, facilitating data reduction without compromising optimization guarantees.

Findings

Effective data screening tests for classification and regression

Reduction of dataset size while maintaining accuracy

Computational gains in empirical risk minimization

Abstract

We design simple screening tests to automatically discard data samples in empirical risk minimization without losing optimization guarantees. We derive loss functions that produce dual objectives with a sparse solution. We also show how to regularize convex losses to ensure such a dual sparsity-inducing property, and propose a general method to design screening tests for classification or regression based on ellipsoidal approximations of the optimal set. In addition to producing computational gains, our approach also allows us to compress a dataset into a subset of representative points.