Obligation and Prohibition Extraction Using Hierarchical RNNs

TL;DR

This paper improves the detection of contractual obligations and prohibitions by introducing a hierarchical BILSTM model with self-attention, outperforming previous methods in accuracy and training speed.

Contribution

It presents a hierarchical BILSTM architecture with self-attention for better discourse-level classification of contractual sentences, surpassing flat models.

Findings

Hierarchical BILSTM outperforms flat models in accuracy.

Self-attention enhances token focus and model performance.

Hierarchical model trains faster and captures broader discourse context.

Abstract

We consider the task of detecting contractual obligations and prohibitions. We show that a self-attention mechanism improves the performance of a BILSTM classifier, the previous state of the art for this task, by allowing it to focus on indicative tokens. We also introduce a hierarchical BILSTM, which converts each sentence to an embedding, and processes the sentence embeddings to classify each sentence. Apart from being faster to train, the hierarchical BILSTM outperforms the flat one, even when the latter considers surrounding sentences, because the hierarchical model has a broader discourse view.