Two Stochastic Optimization Algorithms for Convex Optimization with Fixed Point Constraints

TL;DR

This paper introduces two novel stochastic algorithms for convex optimization problems with fixed point constraints, addressing cases where projections are computationally challenging, and demonstrates their convergence and effectiveness.

Contribution

It proposes two new stochastic algorithms combining fixed point methods with convex optimization techniques, suitable for nonsmooth problems and complex constraints.

Findings

Both algorithms converge almost surely to the solution set.

Convergence rates indicate high efficiency of the methods.

Numerical experiments confirm their practical effectiveness.

Abstract

Two optimization algorithms are proposed for solving a stochastic programming problem for which the objective function is given in the form of the expectation of convex functions and the constraint set is defined by the intersection of fixed point sets of nonexpansive mappings in a real Hilbert space. This setting of fixed point constraints enables consideration of the case in which the projection onto each of the constraint sets cannot be computed efficiently. Both algorithms use a convex function and a nonexpansive mapping determined by a certain probabilistic process at each iteration. One algorithm blends a stochastic gradient method with the Halpern fixed point algorithm. The other is based on a stochastic proximal point algorithm and the Halpern fixed point algorithm; it can be applied to nonsmooth convex optimization. Convergence analysis showed that, under certain assumptions, any weak sequential cluster point of the sequence generated by either algorithm almost surely belongs to the solution set of the problem. Convergence rate analysis illustrated their efficiency, and the numerical results of convex optimization over fixed point sets demonstrated their effectiveness.