Transformer with Implicit Edges for Particle-based Physics Simulation

TL;DR

This paper introduces TIE, a Transformer-based approach that models particle interactions implicitly without explicit edges, reducing computational costs and improving generalization in physics simulations.

Contribution

The paper proposes TIE, a novel Transformer architecture with implicit edges for particle simulation, eliminating explicit graph edges and enhancing efficiency and generalization.

Findings

TIE outperforms GNN-based methods across multiple physics simulation domains.

TIE achieves better generalization to different materials and complex dynamics.

The model reduces computational overhead compared to explicit edge models.

Abstract

Particle-based systems provide a flexible and unified way to simulate physics systems with complex dynamics. Most existing data-driven simulators for particle-based systems adopt graph neural networks (GNNs) as their network backbones, as particles and their interactions can be naturally represented by graph nodes and graph edges. However, while particle-based systems usually contain hundreds even thousands of particles, the explicit modeling of particle interactions as graph edges inevitably leads to a significant computational overhead, due to the increased number of particle interactions. Consequently, in this paper we propose a novel Transformer-based method, dubbed as Transformer with Implicit Edges (TIE), to capture the rich semantics of particle interactions in an edge-free manner. The core idea of TIE is to decentralize the computation involving pair-wise particle interactions into per-particle updates. This is achieved by adjusting the self-attention module to resemble the update formula of graph edges in GNN. To improve the generalization ability of TIE, we further amend TIE with learnable material-specific abstract particles to disentangle global material-wise semantics from local particle-wise semantics. We evaluate our model on diverse domains of varying complexity and materials. Compared with existing GNN-based methods, without bells and whistles, TIE achieves superior performance and generalization across all these domains. Codes and models are available at https://github.com/ftbabi/TIE_ECCV2022.git.