Optimization Condition and Algorithm of Optimization with Convertible Nonconvex Function

TL;DR

This paper introduces the concept of convertible nonconvex functions, establishes conditions for global optimality, and proposes an augmented Lagrangian algorithm with proven convergence for solving complex nonconvex optimization problems.

Contribution

It defines new classes of convertible nonconvex functions, proves duality and optimality conditions, and develops a convergent augmented Lagrangian algorithm for nonconvex optimization.

Findings

Convertible nonconvex functions include many nonsmooth functions.

Strong duality holds for differentiable convertible nonconvex functions.

The proposed algorithm converges for unconstrained nonconvex problems.

Abstract

The paper introduces several new concepts for solving nonconvex or nonsmooth optimization problems, including convertible nonconvex function, exact convertible nonconvex function and differentiable convertible nonconvex function. It is proved herein many nonconvex functions or nonsmooth (or discontinuous) functions are actually convertible nonconvex functions and convertible nonconvex function operations such as addition, subtraction, multiplication or division result in convertible nonconvex functions. The sufficient condition for judging a global optimal solution to unconstrained optimization problems with differentiable convertible nonconvex functions is proved, which is equivalent to Karush-Kuhn-Tucker(KKT) condition. Two Lagrange functions of differentiable convertible nonconvex function are defined with their dual problems defined accordingly. The strong duality theorem is proved, showing that the optimal objective value of the global optimal solution is equal to the optimal objective value of the dual problem, which is equivalent to KKT condition. An augmented Lagrangian penalty function algorithm is proposed and its convergence is proved. So the paper provides a new idea for solving unconstrained nonconvex or non-smooth optimization problems and avoids subdifferentiation or smoothing techniques by using some gradient search algorithms, such as gradient descent algorithm, Newton algorithm and so on.