Hyperparameter optimization of orthogonal functions in the numerical solution of differential equations

TL;DR

This paper develops parallel grid and random search algorithms for hyperparameter optimization of rational Jacobi functions in spectral methods, improving the numerical solution of differential equations.

Contribution

It introduces a hyperparameter optimization framework for spectral methods using rational Jacobi functions, enhancing stability and convergence in solving differential equations.

Findings

Optimized hyperparameters improve solution accuracy.

Parallel algorithms reduce computation time.

Sensitivity analysis informs stability and convergence.

Abstract

This paper considers the hyperparameter optimization problem of mathematical techniques that arise in the numerical solution of differential and integral equations. The well-known approaches grid and random search, in a parallel algorithm manner, are developed to find the optimal set of hyperparameters. Employing rational Jacobi functions, we ran these algorithms on two nonlinear benchmark differential equations on the semi-infinite domain. The configurations contain different rational mappings along with their length scale parameter and the Jacobi functions parameters. These trials are configured on the collocation Least-Squares Support Vector Regression (CLS-SVR), a novel numerical simulation approach based on spectral methods. In addition, we have addressed the sensitivity of these hyperparameters on the numerical stability and convergence of the CLS-SVR model. The experiments show that this technique can effectively improve state-of-the-art results.