MolSpectra: Pre-training 3D Molecular Representation with Multi-modal Energy Spectra

TL;DR

MolSpectra introduces a quantum mechanics-inspired pre-training method for 3D molecular representations using energy spectra, improving molecular property prediction and dynamics modeling.

Contribution

The paper presents SpecFormer, a multi-spectrum encoder that incorporates quantum mechanical energy spectra into 3D molecular representation learning.

Findings

Outperforms existing methods on molecular property prediction benchmarks.

Effectively models quantum mechanical effects in molecular energy states.

Enhances understanding of molecular dynamics through spectral information.

Abstract

Establishing the relationship between 3D structures and the energy states of molecular systems has proven to be a promising approach for learning 3D molecular representations. However, existing methods are limited to modeling the molecular energy states from classical mechanics. This limitation results in a significant oversight of quantum mechanical effects, such as quantized (discrete) energy level structures, which offer a more accurate estimation of molecular energy and can be experimentally measured through energy spectra. In this paper, we propose to utilize the energy spectra to enhance the pre-training of 3D molecular representations (MolSpectra), thereby infusing the knowledge of quantum mechanics into the molecular representations. Specifically, we propose SpecFormer, a multi-spectrum encoder for encoding molecular spectra via masked patch reconstruction. By further aligning outputs from the 3D encoder and spectrum encoder using a contrastive objective, we enhance the 3D encoder's understanding of molecules. Evaluations on public benchmarks reveal that our pre-trained representations surpass existing methods in predicting molecular properties and modeling dynamics.