A Multi-level procedure for enhancing accuracy of machine learning algorithms

TL;DR

This paper introduces a multi-level machine learning approach that combines coarse and fine resolution data to improve accuracy and efficiency in scientific computing applications, especially for differential equations.

Contribution

The paper presents a novel multi-level algorithm that reduces generalization error and computational cost by integrating data at multiple resolutions, outperforming single-level methods.

Findings

Significant accuracy improvements over single-level algorithms.

Notable computational speed-up in uncertainty quantification tasks.

Theoretical support for variance reduction in the proposed method.

Abstract

We propose a multi-level method to increase the accuracy of machine learning algorithms for approximating observables in scientific computing, particularly those that arise in systems modeled by differential equations. The algorithm relies on judiciously combining a large number of computationally cheap training data on coarse resolutions with a few expensive training samples on fine grid resolutions. Theoretical arguments for lowering the generalization error, based on reducing the variance of the underlying maps, are provided and numerical evidence, indicating significant gains over underlying single-level machine learning algorithms, are presented. Moreover, we also apply the multi-level algorithm in the context of forward uncertainty quantification and observe a considerable speed-up over competing algorithms.