Summarisation of German Judgments in conjunction with a Class-based Evaluation

TL;DR

This paper explores automated summarisation of German legal judgments using a fine-tuned large language model, incorporating legal entities to improve content relevance, but finds the quality still needs enhancement for practical use.

Contribution

It introduces a method for summarising German judgments with legal entity enrichment and a class-based evaluation framework for summary quality assessment.

Findings

Legal entity enrichment improves relevance of summaries

Summary quality is currently insufficient for practical deployment

Proposed evaluation classes measure language, pertinence, completeness, correctness

Abstract

The automated summarisation of long legal documents can be a great aid for legal experts in their daily work. We automatically create summaries (guiding principles) of German judgments by fine-tuning a decoder-based large language model. We enrich the judgments with information about legal entities before the training. For the evaluation of the created summaries, we define a set of evaluation classes which allows us to measure their language, pertinence, completeness and correctness. Our results show that employing legal entities helps the generative model to find the relevant content, but the quality of the created summaries is not yet sufficient for a use in practice.