Classifying long legal documents using short random chunks

TL;DR

This paper introduces a method for classifying long legal documents by using randomly selected short chunks with a DeBERTa V3 and LSTM model, combined with a robust deployment pipeline, achieving high accuracy and efficient processing.

Contribution

It presents a novel approach of classifying lengthy legal texts using random chunks and a combined DeBERTa V3 and LSTM model, along with a durable deployment pipeline.

Findings

Weighted F-score of 0.898 achieved.

Median processing time of 498 seconds per 100 files on CPU.

Robust deployment pipeline using Temporal.

Abstract

Classifying legal documents is a challenge, besides their specialized vocabulary, sometimes they can be very long. This means that feeding full documents to a Transformers-based models for classification might be impossible, expensive or slow. Thus, we present a legal document classifier based on DeBERTa V3 and a LSTM, that uses as input a collection of 48 randomly-selected short chunks (max 128 tokens). Besides, we present its deployment pipeline using Temporal, a durable execution solution, which allow us to have a reliable and robust processing workflow. The best model had a weighted F-score of 0.898, while the pipeline running on CPU had a processing median time of 498 seconds per 100 files.