Challenges and Thrills of Legal Arguments

TL;DR

This paper introduces HumBERT, an extension of transformer models, designed to improve inter-sequence attention for conversation-like scenarios in legal argument generation.

Contribution

HumBERT is a novel transformer extension that enables continuous contextual argument generation in conversation scenarios, addressing limitations of existing attention mechanisms.

Findings

HumBERT effectively models inter-sequence attention in legal conversations.

It improves the coherence of generated legal arguments.

The approach demonstrates potential for enhanced legal AI applications.

Abstract

State-of-the-art attention based models, mostly centered around the transformer architecture, solve the problem of sequence-to-sequence translation using the so-called scaled dot-product attention. While this technique is highly effective for estimating inter-token attention, it does not answer the question of inter-sequence attention when we deal with conversation-like scenarios. We propose an extension, HumBERT, that attempts to perform continuous contextual argument generation using locally trained transformers.