Image-to-Graph Transformers for Chemical Structure Recognition

TL;DR

This paper introduces an image-to-graph transformer model that directly converts molecular images into graph structures, improving robustness and accuracy in chemical structure recognition from diverse image styles.

Contribution

The novel model transforms molecular images into graphs end-to-end, handling abbreviations and style variations, outperforming existing methods on benchmark datasets.

Findings

Achieved 17.1% relative improvement on benchmark datasets.

Achieved 12.8% relative improvement on large molecular images.

Demonstrated robustness to diverse image styles.

Abstract

For several decades, chemical knowledge has been published in written text, and there have been many attempts to make it accessible, for example, by transforming such natural language text to a structured format. Although the discovered chemical itself commonly represented in an image is the most important part, the correct recognition of the molecular structure from the image in literature still remains a hard problem since they are often abbreviated to reduce the complexity and drawn in many different styles. In this paper, we present a deep learning model to extract molecular structures from images. The proposed model is designed to transform the molecular image directly into the corresponding graph, which makes it capable of handling non-atomic symbols for abbreviations. Also, by end-to-end learning approach it can fully utilize many open image-molecule pair data from various sources, and hence it is more robust to image style variation than other tools. The experimental results show that the proposed model outperforms the existing models with 17.1 % and 12.8 % relative improvement for well-known benchmark datasets and large molecular images that we collected from literature, respectively.