Notation-level confounding: When inconsistent molecular notations mislead chemical language models

TL;DR

This paper reveals that inconsistent molecular notations, especially in SMILES representations, can mislead chemical language models by distorting their understanding and inflating performance metrics, emphasizing the need for transparency in preprocessing.

Contribution

It introduces the concept of notation-level confounding in CLMs and demonstrates its impact through a survey and molecular translation framework, highlighting the importance of standardized notation reporting.

Findings

Approximately half of surveyed papers did not specify canonicalization procedures.

Inconsistent notations can distort latent representations in CLMs.

Misleading performance gains can result from notation-level confounding.

Abstract

Chemical language models (CLMs) are increasingly used for molecular design and property prediction. Because these models learn from textual encodings of molecules, differences in how such encodings are generated may affect their behavior. In cheminformatics, the term canonical SMILES implies a single standardized notation, yet different toolkits define distinct canonicalization rules, yielding multiple canonical strings for the same molecule. To examine how this variability arises and why it matters, we surveyed 264 CLM papers in PubMed and found that about half did not specify their canonicalization procedure, limiting transparency and reproducibility. Using a molecular translation framework, we show that when multiple valid notations are mixed or left undocumented, inconsistent notations distort latent representations and, in some benchmarks, can spuriously inflate predictive accuracy, a phenomenon we term notation-level confounding. These findings demonstrate how subtle differences in SMILES generation can mislead CLMs and highlight the importance of explicitly reporting preprocessing tools and settings.