Avoiding Traps in Nonconvex Problems

TL;DR

This paper investigates how iterative projection methods can get stuck at non-solutions in nonconvex problems and explores parameter tuning strategies to prevent this, supported by illustrative examples.

Contribution

It introduces the distinction between hyperparameters and metric parameters in nonconvex projection methods and demonstrates their roles in avoiding traps.

Findings

Proper tuning of hyperparameters prevents trapping at non-solutions.

Adjusting metric parameters in constraint sets improves convergence.

Heuristic guidelines for parameter selection are provided.

Abstract

Iterative projection methods may become trapped at non-solutions when the constraint sets are nonconvex. Two kinds of parameters are available to help avoid this behavior and this study gives examples of both. The first kind of parameter, called a hyperparameter, includes any kind of parameter that appears in the definition of the iteration rule itself. The second kind comprises metric parameters in the definition of the constraint sets, a feature that arises when the problem to be solved has two or more kinds of variables. Through examples we show the importance of properly tuning both kinds of parameters and offer heuristic interpretations of the observed behavior.