Exploring Hierarchical Molecular Graph Representation in Multimodal LLMs

TL;DR

This paper investigates the impact of multi-level graph features on multimodal LLMs for biochemical tasks, revealing that hierarchical graph representations enhance model understanding and performance.

Contribution

It introduces an analysis of feature granularity effects and highlights the need for dynamic hierarchical graph processing in multimodal LLMs for molecular data.

Findings

Reducing GNN features to a single token does not harm performance.

Different graph feature levels influence molecule quality and task accuracy.

Current models lack comprehensive understanding of hierarchical graph features.

Abstract

Following the milestones in large language models (LLMs) and multimodal models, we have seen a surge in applying LLMs to biochemical tasks. Leveraging graph features and molecular text representations, LLMs can tackle various tasks, such as predicting chemical reaction outcomes and describing molecular properties. However, most current work overlooks the *multi-level nature* of the graph modality, even though different chemistry tasks may benefit from different feature levels. In this work, we first study the effect of feature granularity and reveal that even reducing all GNN-generated feature tokens to a single one does not significantly impact model performance. We then investigate the effect of various graph feature levels and demonstrate that both the quality of LLM-generated molecules and model performance across different tasks depend on different graph feature levels. Therefore, we conclude with two key insights: (1) current molecular-related multimodal LLMs lack a comprehensive understanding of graph features, and (2) static processing is not sufficient for hierarchical graph feature. We share our findings in detail, with the hope of paving the way for the community to develop more advanced multimodal LLMs for incorporating molecular graphs.