Optimization via Chebyshev Polynomials

TL;DR

This paper introduces a novel robust line-search optimization method using Chebyshev pseudospectral schemes, achieving high accuracy and quadratic convergence through adaptive procedures and detailed error analysis.

Contribution

It presents the first robust exact line-search method based on Chebyshev pseudospectral approximations with adaptive Newton iterations and comprehensive error analysis.

Findings

Achieves quadratic convergence rate.

Demonstrates superior accuracy in numerical experiments.

Effective on multi-dimensional optimization problems.

Abstract

This paper presents for the first time a robust exact line-search method based on a full pseudospectral (PS) numerical scheme employing orthogonal polynomials. The proposed method takes on an adaptive search procedure and combines the superior accuracy of Chebyshev PS approximations with the high-order approximations obtained through Chebyshev PS differentiation matrices (CPSDMs). In addition, the method exhibits quadratic convergence rate by enforcing an adaptive Newton search iterative scheme. A rigorous error analysis of the proposed method is presented along with a detailed set of pseudocodes for the established computational algorithms. Several numerical experiments are conducted on one- and multi-dimensional optimization test problems to illustrate the advantages of the proposed strategy.