On nondegenerate M-stationary points for sparsity constrained nonlinear optimization

TL;DR

This paper analyzes the topological structure of sparsity constrained nonlinear optimization, introducing nondegenerate M-stationary points and revealing their complex role in the global optimization landscape.

Contribution

It introduces the concept of nondegenerate M-stationary points, defines their M-index, and applies Morse theory to understand the global structure of SCNO, highlighting the complexity of saddle points.

Findings

All M-stationary points are generically nondegenerate.

Local minimizers always have active sparsity constraints.

Saddle points can lead to multiple local minimizers.

Abstract

We study sparsity constrained nonlinear optimization (SCNO) from a topological point of view. Special focus will be on M-stationary points from Burdakov et al. (2016). We introduce nondegenerate M-stationary points and define their M-index. We show that all M-stationary points are generically nondegenerate. In particular, the sparsity constraint is active at all local minimizers of a generic SCNO. Some relations to other stationarity concepts, such as S-stationarity, basic feasibility, and CW-minimality, are discussed in detail. By doing so, the issues of instability and degeneracy of points due to different stationarity concepts are highlighted. The concept of M-stationarity allows to adequately describe the global structure of SCNO along the lines of Morse theory. For that, we study topological changes of lower level sets while passing an M-stationary point. As novelty for SCNO, multiple cells of dimension equal to the M-index are needed to be attached. This intriguing fact is in strong contrast with other optimization problems considered before, where just one cell suffices. As a consequence, we derive a Morse relation for SCNO, which relates the numbers of local minimizers and M-stationary points of M-index equal to one. The appearance of such saddle points cannot be thus neglected from the perspective of global optimization. Due to the multiplicity phenomenon in cell-attachment, a saddle point may lead to more than two different local minimizers. We conclude that the relatively involved structure of saddle points is the source of well-known difficulty if solving SCNO to global optimality.