Minimizing a Sum of Clipped Convex Functions

TL;DR

This paper addresses the NP-hard problem of minimizing sums of clipped convex functions by proposing heuristics and an alternative formulation that enables bounds and certification of solution quality, with practical applications and open-source code.

Contribution

It introduces heuristics for approximate solutions and a perspective transformation-based formulation for bounds and solution certification.

Findings

Heuristics find good solutions in practice.

Perspective transformation provides computationally tractable lower bounds.

Solutions are relatively close to the global optimum.

Abstract

We consider the problem of minimizing a sum of clipped convex functions; applications include clipped empirical risk minimization and clipped control. While the problem of minimizing the sum of clipped convex functions is NP-hard, we present some heuristics for approximately solving instances of these problems. These heuristics can be used to find good, if not global, solutions and appear to work well in practice. We also describe an alternative formulation, based on the perspective transformation, which makes the problem amenable to mixed-integer convex programming and yields computationally tractable lower bounds. We illustrate one of our heuristic methods by applying it to various examples and use the perspective transformation to certify that the solutions are relatively close to the global optimum. This paper is accompanied by an open-source implementation.