Conic Descent Redux for Memory-Efficient Optimization

TL;DR

This paper enhances the conic descent method for optimization by providing geometric intuition, theoretical insights, and a memory-efficient algorithm, improving scalability and convergence in signal processing and machine learning tasks.

Contribution

It introduces a momentum variant of conic descent, offers a dual-based geometric derivation, and develops a memory-efficient algorithm for large-scale semidefinite programming.

Findings

Momentum conic descent (MOCO) accelerates convergence.

Memory-efficient MOCO scales SDP for low-rank solutions.

Analytically justified stopping criterion improves reliability.

Abstract

Conic programming has well-documented merits in a gamut of signal processing and machine learning tasks. This contribution revisits a recently developed first-order conic descent (CD) solver, and advances it in three aspects: intuition, theory, and algorithmic implementation. It is found that CD can afford an intuitive geometric derivation that originates from the dual problem. This opens the door to novel algorithmic designs, with a momentum variant of CD, momentum conic descent (MOCO) exemplified. Diving deeper into the dual behavior CD and MOCO reveals: i) an analytically justified stopping criterion; and, ii) the potential to design preconditioners to speed up dual convergence. Lastly, to scale semidefinite programming (SDP) especially for low-rank solutions, a memory efficient MOCO variant is developed and numerically validated.