Do the Benefits of Joint Models for Relation Extraction Extend to Document-level Tasks?

TL;DR

This paper evaluates whether joint models for relation extraction, which outperform pipeline models at sentence level, are effective at document level, finding their advantages diminish in more complex, long-range tasks.

Contribution

It benchmarks state-of-the-art joint and pipeline models on document-level relation extraction, revealing limitations of joint models in long-range, multi-sentence contexts.

Findings

Joint models outperform pipeline models at sentence level.

Performance of joint models drops significantly at document level.

Pipeline models remain more effective for document-level extraction.

Abstract

Two distinct approaches have been proposed for relational triple extraction - pipeline and joint. Joint models, which capture interactions across triples, are the more recent development, and have been shown to outperform pipeline models for sentence-level extraction tasks. Document-level extraction is a more challenging setting where interactions across triples can be long-range, and individual triples can also span across sentences. Joint models have not been applied for document-level tasks so far. In this paper, we benchmark state-of-the-art pipeline and joint extraction models on sentence-level as well as document-level datasets. Our experiments show that while joint models outperform pipeline models significantly for sentence-level extraction, their performance drops sharply below that of pipeline models for the document-level dataset.