# An equivariant pretrained transformer for unified 3D molecular representation learning

## TL;DR

This paper introduces a 3D molecular model that learns from multiple domains to predict molecular properties and help discover antiviral compounds.

## Contribution

The novel contribution is an equivariant transformer model for unified 3D molecular representation across domains.

## Key findings

- The model achieves strong performance in ligand binding affinity prediction.
- It performs competitively in predicting properties of proteins and small molecules.
- The model identifies potential antiviral compounds against the main protease of the COVID-19 virus.

## Abstract

Pretraining on a large number of unlabeled 3D molecules has showcased superiority in various scientific applications. However, prior efforts typically focus on pretraining models in a specific domain, missing the opportunity to leverage cross-domain knowledge. To mitigate this gap, we introduce Equivariant Pretrained Transformer, an all-atom foundation model that can be pretrained from multiple domain 3D molecules. Built upon an E(3)-equivariant transformer, the model learns both atom-level interactions and graph-level structural features (e.g. residuals in proteins), allowing it to generalize across diverse tasks. The model achieves strong gains in ligand binding affinity prediction, while also performing competitively in predicting properties of proteins and small molecules. We further show that the model can help identify potential antiviral compounds against the main protease of the COVID-19 virus, and validate promising candidates through computational and experimental studies.

The study presents a 3D molecular foundation model trained across diverse biological domains to accurately predict properties of proteins and small molecules and aid in the discovery of potential antiviral compounds.