Screening Rules for Convex Problems

TL;DR

This paper introduces a comprehensive framework for deriving screening rules in convex optimization, enabling efficient identification of irrelevant variables across various constrained and penalized problems.

Contribution

It presents a general two-step framework that leverages approximate solutions and duality gaps to generate safe screening rules for a wide range of convex problems.

Findings

Developed new screening rules for simplex and L1-constrained problems

Extended screening techniques to Elastic Net and SVMs

Unified approach applicable to structured norm regularized problems

Abstract

We propose a new framework for deriving screening rules for convex optimization problems. Our approach covers a large class of constrained and penalized optimization formulations, and works in two steps. First, given any approximate point, the structure of the objective function and the duality gap is used to gather information on the optimal solution. In the second step, this information is used to produce screening rules, i.e. safely identifying unimportant weight variables of the optimal solution. Our general framework leads to a large variety of useful existing as well as new screening rules for many applications. For example, we provide new screening rules for general simplex and $L_1$-constrained problems, Elastic Net, squared-loss Support Vector Machines, minimum enclosing ball, as well as structured norm regularized problems, such as group lasso.