Principled Acceleration of Iterative Numerical Methods Using Machine Learning

TL;DR

This paper introduces a new framework to analyze and improve machine learning-based acceleration of iterative numerical methods, addressing limitations of existing meta-learning approaches with theoretical guarantees and practical demonstrations.

Contribution

It provides a systematic analysis distinguishing new methods from classical meta-learning, and proposes a novel training approach with proven improvements and versatile applications.

Findings

The new training method outperforms existing approaches.

Theoretical proof of improved convergence.

Demonstrated effectiveness across multiple numerical problems.

Abstract

Iterative methods are ubiquitous in large-scale scientific computing applications, and a number of approaches based on meta-learning have been recently proposed to accelerate them. However, a systematic study of these approaches and how they differ from meta-learning is lacking. In this paper, we propose a framework to analyze such learning-based acceleration approaches, where one can immediately identify a departure from classical meta-learning. We show that this departure may lead to arbitrary deterioration of model performance. Based on our analysis, we introduce a novel training method for learning-based acceleration of iterative methods. Furthermore, we theoretically prove that the proposed method improves upon the existing methods, and demonstrate its significant advantage and versatility through various numerical applications.