Text-based NP Enrichment

TL;DR

This paper introduces a new task called text-based NP enrichment (TNE) that aims to identify all preposition-mediated relations between noun phrases in text, highlighting the challenge for current models and providing a large-scale dataset for future research.

Contribution

The paper formalizes the TNE task, creates the first large-scale dataset, and evaluates language models, revealing significant challenges and opening avenues for further research.

Findings

Fine-tuned models struggle with implicit relations.

The dataset enables comprehensive analysis of NP relations.

Current models have limited performance on TNE.

Abstract

Understanding the relations between entities denoted by NPs in a text is a critical part of human-like natural language understanding. However, only a fraction of such relations is covered by standard NLP tasks and benchmarks nowadays. In this work, we propose a novel task termed text-based NP enrichment (TNE), in which we aim to enrich each NP in a text with all the preposition-mediated relations -- either explicit or implicit -- that hold between it and other NPs in the text. The relations are represented as triplets, each denoted by two NPs related via a preposition. Humans recover such relations seamlessly, while current state-of-the-art models struggle with them due to the implicit nature of the problem. We build the first large-scale dataset for the problem, provide the formal framing and scope of annotation, analyze the data, and report the results of fine-tuned language models on the task, demonstrating the challenge it poses to current technology. A webpage with a data-exploration UI, a demo, and links to the code, models, and leaderboard, to foster further research into this challenging problem can be found at: yanaiela.github.io/TNE/.