The Gradient descent method from the perspective of fractional calculus

TL;DR

This paper introduces gradient descent methods based on $$-fractional derivatives for unconstrained optimization, analyzing their convergence and demonstrating how tunable fractional parameters can enhance performance.

Contribution

It presents a novel fractional calculus approach to gradient methods and introduces an ABM method for solving $$-fractional derivative equations, with convergence analysis.

Findings

Fractional order $$ and weight $$ improve optimization performance.

The proposed methods converge under certain conditions.

Numerical examples validate the tunability benefits.

Abstract

Motivated by gradient methods in optimization theory, we give methods based on $\psi$-fractional derivatives of order $\alpha$ in order to solve unconstrained optimization problems. The convergence of these methods is analyzed in detail. This paper also presents an Adams-Bashforth-Moulton (ABM) method for the estimation of solutions to equations involving $\psi$-fractional derivatives. Numerical examples using the ABM method show that the fractional order $\alpha$ and weight $\psi$ are tunable parameters, which can be helpful for improving the performance of gradient descent methods.