On the superiority of PGMs to PDCAs in nonsmooth nonconvex sparse regression

TL;DR

This paper demonstrates that proximal gradient methods (PGMs) outperform proximal DC algorithms (PDCAs) in nonsmooth nonconvex sparse regression, converging to stronger stationary points with better solution quality and efficiency.

Contribution

It proves that PGMs can converge to d-stationary points in general nonsmooth nonconvex problems without modifications, extending previous results for DC problems.

Findings

PGMs outperform PDCAs in solution quality and computation time.

Modified PGMs like GIST achieve convergence to d-stationary points.

Numerical results confirm the superiority of PGMs in sparse regression tasks.

Abstract

This paper conducts a comparative study of proximal gradient methods (PGMs) and proximal DC algorithms (PDCAs) for sparse regression problems which can be cast as Difference-of-two-Convex-functions (DC) optimization problems. It has been shown that for DC optimization problems, both General Iterative Shrinkage and Thresholding algorithm (GIST), a modified version of PGM, and PDCA converge to critical points. Recently some enhanced versions of PDCAs are shown to converge to d-stationary points, which are stronger necessary condition for local optimality than critical points. In this paper we claim that without any modification, PGMs converge to a d-stationary point not only to DC problems but also to more general nonsmooth nonconvex problems under some technical assumptions. While the convergence to d-stationary points is known for the case where the step size is small enough, the finding of this paper is valid also for extended versions such as GIST and its alternating optimization version, which is to be developed in this paper. Numerical results show that among several algorithms in the two categories, modified versions of PGM perform best among those not only in solution quality but also in computation time.