Multi-Grid Graph Neural Networks with Self-Attention for Computational Mechanics

TL;DR

This paper presents a novel GNN model with self-attention for computational mechanics, significantly improving accuracy and efficiency in CFD simulations, and introduces a new dataset and training method.

Contribution

It introduces a self-attention based GNN model with multigrid techniques and a BERT-inspired self-supervised training method for CFD, outperforming existing models.

Findings

Achieved 15% reduction in RMSE on flow past a cylinder benchmark.

Developed a dynamic mesh pruning technique improving multigrid GNN performance.

Implemented a self-supervised BERT-based training method reducing RMSE by 25%.

Abstract

Advancement in finite element methods have become essential in various disciplines, and in particular for Computational Fluid Dynamics (CFD), driving research efforts for improved precision and efficiency. While Convolutional Neural Networks (CNNs) have found success in CFD by mapping meshes into images, recent attention has turned to leveraging Graph Neural Networks (GNNs) for direct mesh processing. This paper introduces a novel model merging Self-Attention with Message Passing in GNNs, achieving a 15\% reduction in RMSE on the well known flow past a cylinder benchmark. Furthermore, a dynamic mesh pruning technique based on Self-Attention is proposed, that leads to a robust GNN-based multigrid approach, also reducing RMSE by 15\%. Additionally, a new self-supervised training method based on BERT is presented, resulting in a 25\% RMSE reduction. The paper includes an ablation study and outperforms state-of-the-art models on several challenging datasets, promising advancements similar to those recently achieved in natural language and image processing. Finally, the paper introduces a dataset with meshes larger than existing ones by at least an order of magnitude. Code and Datasets will be released at https://github.com/DonsetPG/multigrid-gnn.