Invex Programs: First Order Algorithms and Their Convergence

TL;DR

This paper introduces new first-order algorithms for invex optimization problems, including constrained cases, providing convergence guarantees and rates, thus advancing the solution methods for this special class of non-convex problems.

Contribution

The paper proposes the first algorithms with convergence guarantees for constrained invex problems, extending first-order methods to this class.

Findings

Algorithms with proven convergence rates for invex problems

First to address constrained invex optimization with guarantees

Comparative analysis showing advantages over existing methods

Abstract

Invex programs are a special kind of non-convex problems which attain global minima at every stationary point. While classical first-order gradient descent methods can solve them, they converge very slowly. In this paper, we propose new first-order algorithms to solve the general class of invex problems. We identify sufficient conditions for convergence of our algorithms and provide rates of convergence. Furthermore, we go beyond unconstrained problems and provide a novel projected gradient method for constrained invex programs with convergence rate guarantees. We compare and contrast our results with existing first-order algorithms for a variety of unconstrained and constrained invex problems. To the best of our knowledge, our proposed algorithm is the first algorithm to solve constrained invex programs.