# Stochastic Three Points Method for Unconstrained Smooth Minimization

## TL;DR

This paper introduces a new randomized derivative-free optimization algorithm called the stochastic three points (STP) method for unconstrained smooth minimization, analyzing its iteration complexity under various settings and assumptions.

## Contribution

The paper proposes the STP method, a novel randomized derivative-free algorithm that works with mild assumptions on the search distribution and analyzes its convergence for different convexity scenarios.

## Key findings

- STP method converges under broad distribution assumptions.
- The algorithm is effective for non-convex, convex, and strongly convex functions.
- Various stepsize schemes are analyzed for the method.

## Abstract

In this paper we consider the unconstrained minimization problem of a smooth function in ${\mathbb{R}}^n$ in a setting where only function evaluations are possible. We design a novel randomized derivative-free algorithm --- the stochastic three points (STP) method --- and analyze its iteration complexity. At each iteration, STP generates a random search direction according to a certain fixed probability law. Our assumptions on this law are very mild: roughly speaking, all laws which do not concentrate all measure on any halfspace passing through the origin will work. For instance, we allow for the uniform distribution on the sphere and also distributions that concentrate all measure on a positive spanning set. Given a current iterate $x$, STP compares the objective function at three points: $x$, $x+\alpha s$ and $x-\alpha s$, where $\alpha>0$ is a stepsize parameter and $s$ is the random search direction. The best of these three points is the next iterate. We analyze the method STP under several stepsize selection schemes (fixed, decreasing, estimated through finite differences, etc). We study non-convex, convex and strongly convex cases.