JudgeMeNot: Personalizing Large Language Models to Emulate Judicial Reasoning in Hebrew

TL;DR

This paper presents a method for personalizing large language models to emulate individual judges' reasoning in Hebrew, using synthetic data and instruction tuning, achieving results comparable to human judgment.

Contribution

Introduces a synthetic-organic supervision pipeline for efficient personalization of language models to replicate judicial reasoning in low-resource settings.

Findings

Causal Language Modeling plus instruction tuning outperforms other personalization methods.

Generated outputs are indistinguishable from human judges' reasoning.

Significant improvements in lexical, stylistic, and semantic similarity.

Abstract

Despite significant advances in large language models, personalizing them for individual decision-makers remains an open problem. Here, we introduce a synthetic-organic supervision pipeline that transforms raw judicial decisions into instruction-tuning data, enabling parameter-efficient fine-tuning of personalized models for individual judges in low-resource settings. We compare our approach to state-of-the-art personalization techniques across three different tasks and settings. The results show that Causal Language Modeling followed by synthetically generated instruction-tuning significantly outperforms all other baselines, providing significant improvements across lexical, stylistic, and semantic similarity. Notably, our model-generated outputs are indistinguishable from the reasoning of human judges, highlighting the viability of efficient personalization, even in low-resource settings.