Global solutions to folded concave penalized nonconvex learning

TL;DR

This paper introduces a novel global optimization method for folded concave penalized nonconvex learning problems, providing theoretical guarantees and outperforming existing techniques in solution quality.

Contribution

It develops a mixed integer programming reformulation that guarantees global optimality for nonconvex learning problems with SCAD and MCP penalties.

Findings

MIPGO finds globally optimal solutions with theoretical guarantees.

MIPGO outperforms existing methods in solution quality.

The equivalence to quadratic programs enables efficient optimization.

Abstract

This paper is concerned with solving nonconvex learning problems with folded concave penalty. Despite that their global solutions entail desirable statistical properties, they lack optimization techniques that guarantee global optimality in a general setting. In this paper, we show that a class of nonconvex learning problems are equivalent to general quadratic programs. This equivalence facilitates us in developing mixed integer linear programming reformulations, which admit finite algorithms that find a provably global optimal solution. We refer to this reformulation-based technique as the mixed integer programming-based global optimization (MIPGO). To our knowledge, this is the first global optimization scheme with a theoretical guarantee for folded concave penalized nonconvex learning with the SCAD penalty [J. Amer. Statist. Assoc. 96 (2001) 1348-1360] and the MCP penalty [Ann. Statist. 38 (2001) 894-942]. Numerical results indicate a significant outperformance of MIPGO over the state-of-the-art solution scheme, local linear approximation and other alternative solution techniques in literature in terms of solution quality.