Randomized Deep Structured Prediction for Discourse-Level Processing

TL;DR

This paper introduces a randomized inference approach to efficiently apply deep structured prediction with neural encoders to complex discourse-level NLP tasks like argumentation mining, overcoming computational challenges.

Contribution

It proposes a novel randomized inference method that enables scalable deep structured prediction for long texts with complex dependencies, improving over traditional methods.

Findings

Efficiently leverages neural encoders for discourse-level tasks.

Shows effectiveness in argumentation mining with complex structures.

Reduces computational costs of structured inference.

Abstract

Expressive text encoders such as RNNs and Transformer Networks have been at the center of NLP models in recent work. Most of the effort has focused on sentence-level tasks, capturing the dependencies between words in a single sentence, or pairs of sentences. However, certain tasks, such as argumentation mining, require accounting for longer texts and complicated structural dependencies between them. Deep structured prediction is a general framework to combine the complementary strengths of expressive neural encoders and structured inference for highly structured domains. Nevertheless, when the need arises to go beyond sentences, most work relies on combining the output scores of independently trained classifiers. One of the main reasons for this is that constrained inference comes at a high computational cost. In this paper, we explore the use of randomized inference to alleviate this concern and show that we can efficiently leverage deep structured prediction and expressive neural encoders for a set of tasks involving complicated argumentative structures.