Explainable Molecular Property Prediction: Aligning Chemical Concepts with Predictions via Language Models
Zhenzhong Wang, Zehui Lin, Wanyu Lin, Ming Yang, Minggang Zeng, Kay Chen Tan
TL;DR
This paper introduces Lamole, a language model-based framework for explainable molecular property prediction that aligns chemical concepts with model explanations, improving interpretability without sacrificing accuracy.
Contribution
Lamole leverages chemically meaningful string representations and combines attention and gradient analyses with a novel loss to produce concept-aligned explanations.
Findings
Achieves state-of-the-art explanation accuracy improvements of up to 14.3%.
Maintains comparable classification accuracy to existing models.
Demonstrates effectiveness across multiple mutagenicity and toxicity datasets.
Abstract
Providing explainable molecular property predictions is critical for many scientific domains, such as drug discovery and material science. Though transformer-based language models have shown great potential in accurate molecular property prediction, they neither provide chemically meaningful explanations nor faithfully reveal the molecular structure-property relationships. In this work, we develop a framework for explainable molecular property prediction based on language models, dubbed as Lamole, which can provide chemical concepts-aligned explanations. We take a string-based molecular representation -- Group SELFIES -- as input tokens to pretrain and fine-tune our Lamole, as it provides chemically meaningful semantics. By disentangling the information flows of Lamole, we propose combining self-attention weights and gradients for better quantification of each chemically meaningful…
Peer Reviews
Decision·ICLR 2025 Conference Withdrawn Submission
- The paper is well-organized and clearly written. It provides detailed explanations of the methodology, covering both theoretical foundations and practical implementations. - Methodology-wise, it introduces a marginal loss function that aligns the model's explanations with chemists' annotations. This approach aims to improve the faithfulness of the explanations by ensuring they are grounded in expert knowledge, which is valuable for applications requiring interpretability. - The paper condu
The technical novelty is limited, as Lamole is essentially a BERT model with Group SELFIES input and a new loss function. The use of attention weights combined with gradients for explanations lacks proper justification, especially given known issues with attention-based interpretability. The dependence on expert annotations severely limits practical applicability, while the marginal performance improvements don't justify the added complexity. The evaluation lacks comparisons with modern language
1) The paper incorporates language models (LMs) to facilitate scientifically meaningful explanations, potentially promoting advancements in related fields. 2) The paper proposes a novel method that combines attention weights with gradients to capture interactions between functional groups. 3) The paper presents a brief theoretical analysis that bridges the manifold hypothesis with explainable molecular property prediction. 4) The paper introduces a new model, Lamole, and demonstrates the effe
1) The novelty and contributions of this paper are limited. Although it aims to provide chemically meaningful explanations, understand functional group interactions, and faithfully reveal molecular structure-property relationships, it achieves little beyond addressing functional group interactions. Overall, the paper offers limited insights. 2) The proposed method employs gradients and attention mechanisms to capture functional group interactions; however, it lacks in-depth or theoretical analy
- This work sheds light on the important but often overlooked topic of explainability in molecular property prediction. - The comprehensive experiments, including effective visualizations such as attention scores and explanation visualizations, enhance the overall comprehensibility of the paper.
- The baselines presented in Table 1 are outdated, with the most recent one dating back to 2020. While I understand that the baselines for explanation accuracy may be limited, there is room for improvement in the property prediction baselines. - Including a baseline with GROUPSELFIES (incorporating LSTM/Transformer) in Table 1 could strengthen the experimental results.
1. The explainable molecular property prediction is crucial to drug discovery. 2. The paper is well-written and easy to follow.
1. My primary concern centers on the experimental section of the paper. The seven datasets employed are all relatively small, which may not fully demonstrate the scalability or generalizability of the proposed method. To address this, it would be beneficial to extend the testing to include medium-sized and large-sized datasets. 2. The experimental section would benefit from having more advanced baseline comparisons. Given that the proposed method uses Group SELFIES representation, it is essentia
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMachine Learning in Materials Science · Computational Drug Discovery Methods
MethodsALIGN