Online Stochastic DCA with applications to Principal Component Analysis

TL;DR

This paper introduces new stochastic algorithms based on DC programming and DCA for nonconvex, nonsmooth optimization, with applications to PCA, providing convergence analysis and demonstrating their effectiveness.

Contribution

The paper develops novel stochastic algorithms for nonconvex, nonsmooth optimization using DC programming and DCA, extending their application to machine learning problems like PCA.

Findings

Algorithms converge under certain conditions.

Effective in solving PCA-related problems.

Theoretical analysis supports practical applicability.

Abstract

Stochastic algorithms are well-known for their performance in the era of big data. In convex optimization, stochastic algorithms have been studied in depth and breadth. However, the current body of research on stochastic algorithms for nonsmooth, nonconvex optimization is relatively limited. In this paper, we propose new stochastic algorithms based on DC (Difference of Convex functions) programming and DCA (DC Algorithm) - the backbone of nonconvex, nonsmooth optimization. Since most real-world nonconvex programs fall into the framework of DC programming, our proposed methods can be employed in various situations, in which they confront stochastic nature and nonconvexity simultaneously. The convergence analysis of the proposed algorithms is studied intensively with the help of tools from modern convex analysis and martingale theory. Finally, we study several aspects of the proposed algorithms on an important problem in machine learning: the expected problem in Principal Component Analysis.