Accelerating Legacy Numerical Solvers by Non-intrusive Gradient-based Meta-solving

TL;DR

This paper introduces a non-intrusive, gradient-based meta-solving approach to accelerate legacy numerical solvers without modifying existing codes, leveraging machine learning to optimize hyperparameters efficiently.

Contribution

It presents a novel gradient estimation technique that enables the use of machine learning with non-differentiable legacy codes without any code modifications.

Findings

The method outperforms baseline approaches in theoretical and numerical evaluations.

Application to PETSc demonstrates significant acceleration of established solvers.

The approach is compatible with widely used scientific computing software.

Abstract

Scientific computing is an essential tool for scientific discovery and engineering design, and its computational cost is always a main concern in practice. To accelerate scientific computing, it is a promising approach to use machine learning (especially meta-learning) techniques for selecting hyperparameters of traditional numerical methods. There have been numerous proposals to this direction, but many of them require automatic-differentiable numerical methods. However, in reality, many practical applications still depend on well-established but non-automatic-differentiable legacy codes, which prevents practitioners from applying the state-of-the-art research to their own problems. To resolve this problem, we propose a non-intrusive methodology with a novel gradient estimation technique to combine machine learning and legacy numerical codes without any modification. We theoretically and numerically show the advantage of the proposed method over other baselines and present applications of accelerating established non-automatic-differentiable numerical solvers implemented in PETSc, a widely used open-source numerical software library.