COMO: Closed-Loop Optical Molecule Recognition with Minimum Risk Training

TL;DR

COMO introduces a novel closed-loop framework with Minimum Risk Training for optical chemical structure recognition, effectively optimizing molecule-level criteria and outperforming existing methods on multiple benchmarks.

Contribution

The paper proposes COMO, a closed-loop OCSR system using MRT to directly optimize molecule-level objectives, addressing exposure bias and improving recognition accuracy.

Findings

COMO outperforms existing methods on ten benchmarks.

MRT is architecture-agnostic and enhances model training.

The approach requires less training data than previous methods.

Abstract

Optical chemical structure recognition (OCSR) translates molecular images into machine-readable representations like SMILES strings or molecular graphs, but remains challenging in real-world documents due to inexhaustible variations in chemical structures, shorthand conventions, and visual noise. Most existing deep-learning-based approaches rely on teacher forcing with token-level Maximum Likelihood Estimation (MLE). This training paradigm suffers from exposure bias, as models are trained under ground-truth prefixes but must condition on their own previous predictions during inference. Moreover, token-level MLE objectives hinder the optimization towards molecular-level evaluation criteria such as chemical validity and structural similarity. Here we introduce Minimum Risk Training (MRT) to OCSR and propose COMO (Closed-loop Optical Molecule recOgnition), a closed-loop framework that mitigates exposure bias by directly optimizing over molecule-level, non-differentiable objectives, by iteratively sampling and evaluating the model's own predictions. Experiments on ten benchmarks including synthetic and real-world chemical diagrams from patent and scientific literature demonstrate that COMO substantially outperforms existing rule-based and learning-based methods with less training data. Ablation studies further show that MRT is architecture-agnostic, demonstrating its potential for broad application to end-to-end OCSR systems.