First-order algorithms for a class of fractional optimization problems

TL;DR

This paper introduces first-order algorithms for a class of fractional optimization problems involving nonconvex and nonsmooth functions, providing convergence analysis and demonstrating their effectiveness through numerical experiments.

Contribution

The paper develops new first-order algorithms for fractional problems with nonconvex nonsmooth components, establishing convergence and applying them to sparse generalized eigenvalue problems.

Findings

Algorithms converge to critical points under mild conditions.

Global convergence and convergence rates are established.

Numerical experiments show the algorithms' efficiency.

Abstract

We consider in this paper a class of single-ratio fractional minimization problems, in which the numerator part of the objective is the sum of a nonsmooth nonconvex function and a smooth nonconvex function while the denominator part is a nonsmooth convex function. In this work, we first derive its first-order necessary optimality condition, by using the first-order operators of the three functions involved. Then we develop first-order algorithms, namely, the proximity-gradient-subgradient algorithm (PGSA), PGSA with monotone line search (PGSA_ML) and PGSA with nonmonotone line search (PGSA_NL). It is shown that any accumulation point of the sequence generated by them is a critical point of the problem under mild assumptions. Moreover, we establish global convergence of the sequence generated by PGSA or PGSA_ML and analyze its convergence rate, by further assuming the local Lipschitz continuity of the nonsmooth function in the numerator part, the smoothness of the denominator part and the Kurdyka- Lojasiewicz property of the objective. The proposed algorithms are applied to the sparse generalized eigenvalue problem associated with a pair of symmetric positive semidefinite matrices and the corresponding convergence results are obtained according to their general convergence theorems. We perform some preliminary numerical experiments to demonstrate the efficiency of the proposed algorithms