Benchmarking Graphormer on Large-Scale Molecular Modeling Datasets

TL;DR

This paper evaluates the enhanced Graphormer model on large-scale molecular datasets, demonstrating improved accuracy over traditional GNNs and previous models in quantum chemistry and catalyst modeling tasks.

Contribution

It introduces architecture modifications and adaptation strategies for Graphormer, achieving superior performance on 2D and 3D molecular modeling benchmarks.

Findings

Graphormer outperforms vanilla models on molecular datasets.

Achieves lower MAE on PCQM4M quantum chemistry dataset.

Outperforms competitors in Open Catalyst Challenge.

Abstract

This technical note describes the recent updates of Graphormer, including architecture design modifications, and the adaption to 3D molecular dynamics simulation. With these simple modifications, Graphormer could attain better results on large-scale molecular modeling datasets than the vanilla one, and the performance gain could be consistently obtained on 2D and 3D molecular graph modeling tasks. In addition, we show that with a global receptive field and an adaptive aggregation strategy, Graphormer is more powerful than classic message-passing-based GNNs. Empirically, Graphormer could achieve much less MAE than the originally reported results on the PCQM4M quantum chemistry dataset used in KDD Cup 2021. In the meanwhile, it greatly outperforms the competitors in the recent Open Catalyst Challenge, which is a competition track on NeurIPS 2021 workshop, and aims to model the catalyst-adsorbate reaction system with advanced AI models. All codes could be found at https://github.com/Microsoft/Graphormer.