Natural Language Premise Selection: Finding Supporting Statements for Mathematical Text

TL;DR

This paper introduces the natural premise selection task to identify supporting statements in mathematical texts, providing a dataset and analyzing the challenges faced by current NLP models in understanding mathematical discourse.

Contribution

It proposes a new NLP task for mathematical texts, introduces the NL-PS dataset, and evaluates baseline models to highlight interpretation challenges.

Findings

Baseline models struggle with the complexity of mathematical language.

The NL-PS dataset enables evaluation of premise selection methods.

Understanding mathematical discourse remains a significant challenge for NLP.

Abstract

Mathematical text is written using a combination of words and mathematical expressions. This combination, along with a specific way of structuring sentences makes it challenging for state-of-art NLP tools to understand and reason on top of mathematical discourse. In this work, we propose a new NLP task, the natural premise selection, which is used to retrieve supporting definitions and supporting propositions that are useful for generating an informal mathematical proof for a particular statement. We also make available a dataset, NL-PS, which can be used to evaluate different approaches for the natural premise selection task. Using different baselines, we demonstrate the underlying interpretation challenges associated with the task.