Stress Testing BERT Anaphora Resolution Models for Reaction Extraction in Chemical Patents

TL;DR

This paper evaluates the robustness of BERT-based anaphora resolution models for extracting chemical reactions from patents, comparing performance in noise-free and noisy conditions, and explores methods to enhance their noise resilience.

Contribution

It introduces an analysis of BERT-based anaphora resolution models' robustness in chemical patent reaction extraction and proposes strategies to improve their noise tolerance.

Findings

Models perform significantly worse in noisy environments.

Certain noise reduction techniques improve model robustness.

Anaphora resolution is critical for accurate reaction extraction.

Abstract

The high volume of published chemical patents and the importance of a timely acquisition of their information gives rise to automating information extraction from chemical patents. Anaphora resolution is an important component of comprehensive information extraction, and is critical for extracting reactions. In chemical patents, there are five anaphoric relations of interest: co-reference, transformed, reaction associated, work up, and contained. Our goal is to investigate how the performance of anaphora resolution models for reaction texts in chemical patents differs in a noise-free and noisy environment and to what extent we can improve the robustness against noise of the model.