Domain-Adaptation through Synthetic Data: Fine-Tuning Large Language Models for German Law

TL;DR

This paper introduces a synthetic data generation method for fine-tuning large language models to improve their performance in German legal question answering, reducing reliance on costly manual annotations.

Contribution

The authors develop a novel synthetic data approach from German statutes and demonstrate its effectiveness in adapting LLMs for legal reasoning tasks.

Findings

Significant performance improvement over baseline models

High-quality, legally accurate synthetic question-answer pairs

Efficient fine-tuning with synthetic data

Abstract

Large language models (LLMs) often struggle in specialized domains such as legal reasoning due to limited expert knowledge, resulting in factually incorrect outputs or hallucinations. This paper presents an effective method for adapting advanced LLMs to German legal question answering through a novel synthetic data generation approach. In contrast to costly human-annotated resources or unreliable synthetic alternatives, our approach systematically produces high-quality, diverse, and legally accurate question-answer pairs directly from authoritative German statutes. Using rigorous automated filtering methods and parameter-efficient fine-tuning techniques, we demonstrate that LLMs adapted with our synthetic dataset significantly outperform their baseline counterparts on German legal question answering tasks. Our results highlight the feasibility of using carefully designed synthetic data as a robust alternative to manual annotation in high-stakes, knowledge-intensive domains.