Chem4DLLM: 4D Multimodal LLMs for Chemical Dynamics Understanding

TL;DR

This paper introduces Chem4DLLM, a multimodal large language model designed to interpret 4D molecular trajectories, enabling dynamic chemical understanding through natural language explanations of molecular events.

Contribution

It presents ChemDU, a new task for translating 4D molecular data into explanations, along with Chem4DBench dataset and Chem4DLLM model that combines geometric encoding with language modeling.

Findings

Chem4DLLM effectively captures molecular geometry and dynamics.

Chem4DBench provides a benchmark for dynamic chemical explanations.

The approach advances multimodal reasoning in chemical sciences.

Abstract

Existing chemical understanding tasks primarily rely on static molecular representations, limiting their ability to model inherently dynamic phenomena such as bond breaking or conformational changes, which are essential for a chemist to understand chemical reactions. To address this gap, we introduce Chemical Dynamics Understanding (ChemDU), a new task that translates 4D molecular trajectories into interpretable natural-language explanations. ChemDU focuses on fundamental dynamic scenarios, including gas-phase and catalytic reactions, and requires models to reason about key events along molecular trajectories, such as bond formation and dissociation, and to generate coherent, mechanistically grounded narratives. To benchmark this capability, we construct Chem4DBench, the first dataset pairing 4D molecular trajectories with expert-authored explanations across these settings. We further propose Chem4DLLM, a unified model that integrates an equivariant graph encoder with a pretrained large language model to explicitly capture molecular geometry and rotational dynamics. We hope that ChemDU, together with Chem4DBench and Chem4DLLM, will stimulate further research in dynamic chemical understanding and multimodal scientific reasoning.