Smoothed Functional Algorithms for Stochastic Optimization using q-Gaussian Distributions

TL;DR

This paper introduces a new class of smoothed functional algorithms for stochastic optimization using q-Gaussian distributions, unifying existing kernels and demonstrating convergence and effectiveness in simulations.

Contribution

It proposes a novel SF scheme based on q-Gaussian kernels, providing convergence proofs and practical algorithms for constrained stochastic optimization.

Findings

The q-Gaussian kernel encompasses most existing smoothing kernels.

The proposed algorithms converge to stationary points.

Numerical simulations validate the effectiveness of the methods.

Abstract

Smoothed functional (SF) schemes for gradient estimation are known to be efficient in stochastic optimization algorithms, specially when the objective is to improve the performance of a stochastic system. However, the performance of these methods depends on several parameters, such as the choice of a suitable smoothing kernel. Different kernels have been studied in literature, which include Gaussian, Cauchy and uniform distributions among others. This paper studies a new class of kernels based on the q-Gaussian distribution, that has gained popularity in statistical physics over the last decade. Though the importance of this family of distributions is attributed to its ability to generalize the Gaussian distribution, we observe that this class encompasses almost all existing smoothing kernels. This motivates us to study SF schemes for gradient estimation using the q-Gaussian distribution. Using the derived gradient estimates, we propose two-timescale algorithms for optimization of a stochastic objective function in a constrained setting with projected gradient search approach. We prove the convergence of our algorithms to the set of stationary points of an associated ODE. We also demonstrate their performance numerically through simulations on a queuing model.