CompleteRXN: Toward Completing Open Chemical Reaction Databases

TL;DR

CompleteRXN introduces a large-scale benchmark for reaction completion, evaluating models on incomplete chemical reaction datasets to improve their robustness and applicability.

Contribution

The paper presents CompleteRXN, a new benchmark dataset for reaction completion, and evaluates models including a novel constrained decoding method, highlighting current limitations.

Findings

CRB achieves 99.20% accuracy on random split

SynRBL produces plausible completions but lower accuracy

Performance drops significantly on out-of-distribution reactions

Abstract

Chemical reaction datasets such as USPTO suffer from substantial incompleteness, frequently missing byproducts, co-reactants, and stoichiometric coefficients. This limits their applicability and reliability in downstream applications. Here, we introduce CompleteRXN, a large-scale supervised benchmark for reaction completion under realistic missing-data conditions. We construct a dataset of aligned incomplete and atom-balanced reactions by mapping USPTO records to curated mechanistic reactions. We evaluate representative baselines, including a novel encoder-decoder reaction completion model with constrained decoding, the Constrained Reaction Balancer (CRB), and a recent algorithmic method, SynRBL. On our CompleteRXN benchmark, the CRB achieves high performance across splits of increasing difficulty, reaching 99.20% equivalence accuracy on the random split and 91.12% on the extreme out-of-distribution split. SynRBL produces many balanced and chemically plausible completions, but with lower accuracy on the benchmark test splits. Across all methods, performance degrades with increasing incompleteness. We observe a substantial drop when evaluating on reactions outside the benchmark (full uncurated USPTO), highlighting the gap between benchmark performance and practical robustness and motivating future work.