Finite Element Neural Network Interpolation. Part I: Interpretable and Adaptive Discretization for Solving PDEs

TL;DR

The paper introduces FENNI, an interpretable, adaptive neural network framework for solving PDEs that improves training efficiency and accuracy through mesh-based design, multigrid strategies, and variational loss functions.

Contribution

It extends EFENN with a reference element architecture, multigrid training, and 2D rh-adaptivity, enhancing interpretability and efficiency in PDE solutions.

Findings

FENNI achieves high accuracy with fewer parameters.

Multigrid training improves robustness and speed.

Variational loss performs comparably to energy-based losses.

Abstract

We present the Finite Element Neural Network Interpolation (FENNI) framework, a sparse neural network architecture extending previous work on Embedded Finite Element Neural Networks (EFENN) introduced with the Hierarchical Deep-learning Neural Networks (HiDeNN). Due to their mesh-based structure, EFENN requires significantly fewer trainable parameters than fully connected neural networks, with individual weights and biases having a clear interpretation. Our FENNI framework, within the EFENN framework, brings improvements to the HiDeNN approach. First, we propose a reference element-based architecture where shape functions are defined on a reference element, enabling variability in interpolation functions and straightforward use of Gaussian quadrature rules for evaluating the loss function. Second, we propose a pragmatic multigrid training strategy based on the framework's interpretability. Third, HiDeNN's combined rh-adaptivity is extended from 1D to 2D, with a new Jacobian-based criterion for adding nodes combining h- and r-adaptivity. From a deep learning perspective, adaptive mesh behavior through rh-adaptivity and the multigrid approach correspond to transfer learning, enabling FENNI to optimize the network's architecture dynamically during training. The framework's capabilities are demonstrated on 1D and 2D test cases, where its accuracy and computational cost are compared against an analytical solution and a classical FEM solver. On these cases, the multigrid training strategy drastically improves the training stage's efficiency and robustness. Finally, we introduce a variational loss within the EFENN framework, showing that it performs as well as energy-based losses and outperforms residual-based losses. This framework is extended to surrogate modeling over the parametric space in Part II.