A Stochastic Variance Reduced Primal Dual Fixed Point Method For Linearly Constrained Separable Optimization

TL;DR

This paper introduces a novel stochastic variance reduced primal dual fixed point method (SVRG-PDFP) for efficiently solving linearly constrained separable convex optimization problems, with proven convergence rates and practical applications.

Contribution

It combines SVRG with PDFP to create a generalized method for convex problems, providing convergence analysis and demonstrating effectiveness in machine learning and image reconstruction.

Findings

Convergence rate of O(1/k) for general convex functions

Linear convergence rate for strongly convex functions

Effective in machine learning and CT image reconstruction

Abstract

In this paper we combine the stochastic variance reduced gradient (SVRG) method [17] with the primal dual fixed point method (PDFP) proposed in [7] to solve a sum of two convex functions and one of which is linearly composite. This type of problems are typically arisen in sparse signal and image reconstruction. The proposed SVRG-PDFP can be seen as a generalization of Prox-SVRG [37] originally designed for the minimization of a sum of two convex functions. Based on some standard assumptions, we propose two variants, one is for strongly convex objective function and the other is for general convex cases. Convergence analysis shows that the convergence rate of SVRG-PDFP is O(1/k) (here k is the iteration number) for general convex objective function and linear for k strongly convex case. Numerical examples on machine learning and CT image reconstruction are provided to show the effectiveness of the algorithms.