Heterogenous Ensemble of Models for Molecular Property Prediction

TL;DR

This paper presents a heterogenous ensemble approach combining Transformer, GNN, and ResNet models trained on multiple molecular data modalities, achieving state-of-the-art results in a large-scale molecular property prediction challenge.

Contribution

It introduces a novel ensemble method integrating diverse model architectures and data modalities, winning the OGB Large-Scale Challenge 2022.

Findings

Achieved a test MAE of 0.0723 on the PCQM4Mv2 dataset.

Ensembling multiple modalities improves prediction accuracy.

Inference time is under 2 hours for the entire solution.

Abstract

Previous works have demonstrated the importance of considering different modalities on molecules, each of which provide a varied granularity of information for downstream property prediction tasks. Our method combines variants of the recent TransformerM architecture with Transformer, GNN, and ResNet backbone architectures. Models are trained on the 2D data, 3D data, and image modalities of molecular graphs. We ensemble these models with a HuberRegressor. The models are trained on 4 different train/validation splits of the original train + valid datasets. This yields a winning solution to the 2\textsuperscript{nd} edition of the OGB Large-Scale Challenge (2022) on the PCQM4Mv2 molecular property prediction dataset. Our proposed method achieves a test-challenge MAE of $0.0723$ and a validation MAE of $0.07145$. Total inference time for our solution is less than 2 hours. We open-source our code at https://github.com/jfpuget/NVIDIA-PCQM4Mv2.