SALSA: Semantically-Aware Latent Space Autoencoder

TL;DR

SALSA introduces a contrastive autoencoder that learns semantically meaningful molecular representations by respecting structural similarities, improving the quality of latent space for drug discovery applications.

Contribution

The paper proposes SALSA, a transformer autoencoder with a contrastive loss, to learn structurally-aware and semantically continuous molecular representations from SMILES data.

Findings

SALSA produces a latent space that better respects molecular structural similarities.

The contrastive training improves the semantic continuity of the latent space.

SALSA outperforms ablated models in capturing molecular properties.

Abstract

In deep learning for drug discovery, chemical data are often represented as simplified molecular-input line-entry system (SMILES) sequences which allow for straightforward implementation of natural language processing methodologies, one being the sequence-to-sequence autoencoder. However, we observe that training an autoencoder solely on SMILES is insufficient to learn molecular representations that are semantically meaningful, where semantics are defined by the structural (graph-to-graph) similarities between molecules. We demonstrate by example that autoencoders may map structurally similar molecules to distant codes, resulting in an incoherent latent space that does not respect the structural similarities between molecules. To address this shortcoming we propose Semantically-Aware Latent Space Autoencoder (SALSA), a transformer-autoencoder modified with a contrastive task, tailored specifically to learn graph-to-graph similarity between molecules. Formally, the contrastive objective is to map structurally similar molecules (separated by a single graph edit) to nearby codes in the latent space. To accomplish this, we generate a novel dataset comprised of sets of structurally similar molecules and opt for a supervised contrastive loss that is able to incorporate full sets of positive samples. We compare SALSA to its ablated counterparts, and show empirically that the composed training objective (reconstruction and contrastive task) leads to a higher quality latent space that is more 1) structurally-aware, 2) semantically continuous, and 3) property-aware.