Data-Efficient Molecular Generation with Hierarchical Textual Inversion

TL;DR

HI-Mol introduces a hierarchical textual inversion approach for data-efficient molecular generation, leveraging multi-level embeddings to improve low-data molecule modeling and outperform prior methods significantly.

Contribution

The paper proposes a novel hierarchical textual inversion technique for molecular generation that effectively captures multi-level features, enabling high-quality molecule synthesis with limited data.

Findings

HI-Mol outperforms previous methods with 50x less data on QM9.

Multi-level embeddings improve low-shot molecule distribution learning.

Generated molecules are effective for low-shot property prediction.

Abstract

Developing an effective molecular generation framework even with a limited number of molecules is often important for its practical deployment, e.g., drug discovery, since acquiring task-related molecular data requires expensive and time-consuming experimental costs. To tackle this issue, we introduce Hierarchical textual Inversion for Molecular generation (HI-Mol), a novel data-efficient molecular generation method. HI-Mol is inspired by the importance of hierarchical information, e.g., both coarse- and fine-grained features, in understanding the molecule distribution. We propose to use multi-level embeddings to reflect such hierarchical features based on the adoption of the recent textual inversion technique in the visual domain, which achieves data-efficient image generation. Compared to the conventional textual inversion method in the image domain using a single-level token embedding, our multi-level token embeddings allow the model to effectively learn the underlying low-shot molecule distribution. We then generate molecules based on the interpolation of the multi-level token embeddings. Extensive experiments demonstrate the superiority of HI-Mol with notable data-efficiency. For instance, on QM9, HI-Mol outperforms the prior state-of-the-art method with 50x less training data. We also show the effectiveness of molecules generated by HI-Mol in low-shot molecular property prediction.