Detect the Interactions that Matter in Matter: Geometric Attention for Many-Body Systems

TL;DR

This paper introduces a geometric attention mechanism tailored for many-body systems, enabling neural networks to model atomic interactions in continuous space while respecting physical symmetries.

Contribution

It proposes a novel geometric attention variant that captures atomic interactions in Euclidean space and respects physical symmetries, advancing molecular modeling techniques.

Findings

Effective translation of molecular geometry into atomic contributions.

Ability to model global atomic interactions beyond local dependencies.

Demonstrated applicability to many-body systems.

Abstract

Attention mechanisms are developing into a viable alternative to convolutional layers as elementary building block of NNs. Their main advantage is that they are not restricted to capture local dependencies in the input, but can draw arbitrary connections. This unprecedented capability coincides with the long-standing problem of modeling global atomic interactions in molecular force fields and other many-body problems. In its original formulation, however, attention is not applicable to the continuous domains in which the atoms live. For this purpose we propose a variant to describe geometric relations for arbitrary atomic configurations in Euclidean space that also respects all relevant physical symmetries. We furthermore demonstrate, how the successive application of our learned attention matrices effectively translates the molecular geometry into a set of individual atomic contributions on-the-fly.