Stratified Sampling Algorithms for Machine Learning Methods in Solving Two-scale Partial Differential Equations

TL;DR

This paper introduces a stratified sampling algorithm to improve machine learning solutions for two-scale PDEs, effectively addressing the vanishing gradient problem over large, multiscale domains.

Contribution

The study proposes a novel stratified sampling approach specifically designed for multiscale PDEs, enhancing solution accuracy over traditional uniform sampling methods.

Findings

Stratified sampling outperforms uniform sampling in accuracy.

The method effectively addresses the vanishing gradient problem.

Numerical results show improved solution consistency.

Abstract

Partial differential equations (PDEs) with multiple scales or those defined over sufficiently large domains arise in various areas of science and engineering and often present problems when approximating the solutions numerically. Machine learning techniques are a relatively recent method for solving PDEs. Despite the increasing number of machine learning strategies developed to approximate PDEs, many remain focused on relatively small domains. When scaling the equations, a large domain is naturally obtained, especially when the solution exhibits multiscale characteristics. This study examines two-scale equations whose solution structures exhibit distinct characteristics: highly localized in some regions and significantly flat in others. These two regions must be adequately addressed over a large domain to approximate the solution more accurately. We focus on the vanishing gradient problem given by the diminishing gradient zone of the activation function over large domains and propose a stratified sampling algorithm to address this problem. We compare the uniform random classical sampling method over the entire domain and the proposed stratified sampling method. The numerical results confirm that the proposed method yields more accurate and consistent solutions than classical methods.