Gradient Approximation and Multi-Variable Derivative-Free Optimization based on Non-Commutative Maps

TL;DR

This paper introduces a new derivative-free optimization method for multi-variable functions using non-commutative maps to approximate gradients, with theoretical analysis and numerical validation.

Contribution

It presents a novel gradient approximation technique based on non-commutative maps for derivative-free optimization.

Findings

The proposed algorithms effectively approximate gradients without derivatives.

Theoretical properties of the algorithms are rigorously analyzed.

Numerical examples demonstrate the method's practical performance.

Abstract

In this work, multi-variable derivative-free optimization algorithms for unconstrained optimization problems are developed. A novel procedure for approximating the gradient of multi-variable objective functions based on non-commutative maps is introduced. The procedure is based on the construction of an exploration sequence to specify where the objective function is evaluated and the definition of so-called gradient generating functions which are composed with the objective function such that the procedure mimics a gradient descent algorithm. Various theoretical properties of the proposed class of algorithms are investigated and numerical examples are presented.