Fine-grained Intent Classification in the Legal Domain

TL;DR

This paper introduces a new dataset of legal documents with annotated intent phrases and categories, and evaluates transformer models for intent extraction and classification, highlighting the dataset's complexity.

Contribution

It provides a novel annotated dataset for fine-grained intent classification in the legal domain and analyzes transformer model performance on this challenging task.

Findings

Transformer models struggle with fine-grained intent classification.

The dataset is challenging for current NLP models.

Fine-grained intent extraction requires nuanced understanding.

Abstract

A law practitioner has to go through a lot of long legal case proceedings. To understand the motivation behind the actions of different parties/individuals in a legal case, it is essential that the parts of the document that express an intent corresponding to the case be clearly understood. In this paper, we introduce a dataset of 93 legal documents, belonging to the case categories of either Murder, Land Dispute, Robbery, or Corruption, where phrases expressing intent same as the category of the document are annotated. Also, we annotate fine-grained intents for each such phrase to enable a deeper understanding of the case for a reader. Finally, we analyze the performance of several transformer-based models in automating the process of extracting intent phrases (both at a coarse and a fine-grained level), and classifying a document into one of the possible 4 categories, and observe that, our dataset is challenging, especially in the case of fine-grained intent classification.