Global and local approaches for the minimization of a sum of pointwise minima of convex functions

TL;DR

This paper introduces a new framework for minimizing sums of pointwise minima of convex functions, extending existing models, and proposes scalable local algorithms with proven convergence and empirical advantages.

Contribution

It extends the sum of convex minima framework to arbitrary minima, develops a bi-convex reformulation, and introduces relaxed alternating minimization methods with convergence guarantees.

Findings

r-AM outperforms classical AM and DC methods in experiments.

A compact MICP formulation for SMC is proposed, avoiding extra variables.

Local solutions can be certified or improved using the MICP approach.

Abstract

Numerous machine learning and industrial problems can be modeled as the minimization of a sum of $N$ so-called clipped convex functions (SCC), i.e. each term of the sum stems as the pointwise minimum between a constant and a convex function. In this work, we extend this framework to capture more problems of interest. Specifically, we allow each term of the sum to be a pointwise minimum of an arbitrary number of convex functions, called components, turning the objective into a sum of pointwise minima of convex functions (SMC). Problem (SCC) is NP-hard, highlighting an appeal for scalable local heuristics. In this spirit, one can express (SMC) objectives as the difference between two convex functions to leverage the possibility to apply (DC) algorithms to compute critical points of the problem. Our approach relies on a bi-convex reformulation of the problem. From there, we derive a family of local methods, dubbed as relaxed alternating minimization (r-AM) methods, that include classical alternating minimization (AM) as a special case. We prove that every accumulation point of r-AM is critical. In addition, we show the empirical superiority of r-AM, compared to traditional AM and (DC) approaches, on piecewise-linear regression and restricted facility location problems. Under mild assumptions, (SCC) can be cast as a mixed-integer convex program (MICP) using perspective functions. This approach can be generalized to (SMC) but introduces many copies of the primal variable. In contrast, we suggest a compact big-M based (MICP) equivalent formulation of (SMC), free of these extra variables. Finally, we showcase practical examples where solving our (MICP), restricted to a neighbourhood of a given candidate (i.e. output iterate of a local method), will either certify the candidate's optimality on that neighbourhood or providing a new point, strictly better, to restart the local method.