Exact Instance Compression for Convex Empirical Risk Minimization via Color Refinement

TL;DR

This paper introduces a lossless compression framework for convex empirical risk minimization problems using color refinement, significantly improving computational efficiency across various models.

Contribution

It extends color refinement techniques to a broad class of convex ERM problems, providing concrete algorithms for multiple models and demonstrating their effectiveness.

Findings

Reduces computational cost of convex ERM

Applicable to diverse models like logistic and kernel regression

Shows strong empirical performance on datasets

Abstract

Empirical risk minimization (ERM) can be computationally expensive, with standard solvers scaling poorly even in the convex setting. We propose a novel lossless compression framework for convex ERM based on color refinement, extending prior work from linear programs and convex quadratic programs to a broad class of differentiable convex optimization problems. We develop concrete algorithms for a range of models, including linear and polynomial regression, binary and multiclass logistic regression, regression with elastic-net regularization, and kernel methods such as kernel ridge regression and kernel logistic regression. Numerical experiments on representative datasets demonstrate the effectiveness of the proposed approach.