Beyond Chemical 1D knowledge using Transformers

TL;DR

This paper evaluates Transformer-CNN models for predicting chemical properties from augmented SMILES, comparing stereochemistry representations, and finds models perform well even without stereochemical info, advancing NLP in chemical 3D tasks.

Contribution

It demonstrates that Transformer-CNN models can effectively predict chemical properties using simplified SMILES representations without stereochemistry, challenging the necessity of detailed stereochemical encoding.

Findings

Models perform well without stereochemistry in SMILES.

Transformer-CNN outperforms descriptor-based models on 3D structure tasks.

Stereochemical encoding does not always improve model performance.

Abstract

In the present paper we evaluated efficiency of the recent Transformer-CNN models to predict target properties based on the augmented stereochemical SMILES. We selected a well-known Cliff activity dataset as well as a Dipole moment dataset and compared the effect of three representations for R/S stereochemistry in SMILES. The considered representations were SMILES without stereochemistry (noChiSMI), classical relative stereochemistry encoding (RelChiSMI) and an alternative version with absolute stereochemistry encoding (AbsChiSMI). The inclusion of R/S in SMILES representation allowed simplify the assignment of the respective information based on SMILES representation, but did not always show advantages on regression or classification tasks. Interestingly, we did not see degradation of the performance of Transformer-CNN models when the stereochemical information was not present in SMILES. Moreover, these models showed higher or similar performance compared to descriptor-based models based on 3D structures. These observations are an important step in NLP modeling of 3D chemical tasks. An open challenge remains whether Transformer-CNN can efficiently embed 3D knowledge from SMILES input and whether a better representation could further increase the accuracy of this approach.