Convergence Rates of Finite Difference Stochastic Approximation Algorithms

TL;DR

This paper analyzes how finite difference methods affect the convergence rates of derivative-free stochastic optimization algorithms, showing potential acceleration to optimal rates under certain schemes.

Contribution

It provides theoretical analysis of convergence rates for Kiefer-Wolfowitz and mirror descent algorithms using finite differences, highlighting ways to accelerate convergence.

Findings

Convergence rate can be improved to n^{-2/5} generally.

In Monte Carlo optimization, rate can reach n^{-1/2}.

Finite difference implementation controls significantly impact convergence speed.

Abstract

Recently there has been renewed interests in derivative free approaches to stochastic optimization. In this paper, we examine the rates of convergence for the Kiefer-Wolfowitz algorithm and the mirror descent algorithm, under various updating schemes using finite differences as gradient approximations. It is shown that the convergence of these algorithms can be accelerated by controlling the implementation of the finite differences. Particularly, it is shown that the rate can be increased to $n^{-2/5}$ in general and to $n^{-1/2}$ in Monte Carlo optimization for a broad class of problems, in the iteration number n.