Text to Trust: Evaluating Fine-Tuning and LoRA Trade-offs in Language Models for Unfair Terms of Service Detection

TL;DR

This paper systematically compares fine-tuning, LoRA, and zero-shot prompting for detecting unfair clauses in legal Terms of Service documents, highlighting trade-offs between accuracy and resource efficiency.

Contribution

It provides a comprehensive evaluation of adaptation strategies for legal NLP tasks, introducing open baselines and analyzing practical trade-offs.

Findings

Full fine-tuning yields the best precision-recall balance.

LoRA models achieve competitive recall with significantly lower memory usage.

Zero-shot GPT-4o performs well but with less domain-specific accuracy.

Abstract

Large Language Models (LLMs) have transformed text understanding, yet their adaptation to specialized legal domains remains constrained by the cost of full fine-tuning. This study provides a systematic evaluation of fine tuning, parameter efficient adaptation (LoRA, QLoRA), and zero-shot prompting strategies for unfair clause detection in Terms of Service (ToS) documents, a key application in legal NLP. We finetune BERT and DistilBERT, apply 4-bit Low-Rank Adaptation (LoRA) to models such as TinyLlama, LLaMA 3B/7B, and SaulLM, and evaluate GPT-4o and O-versions in zero-shot settings. Experiments on the CLAUDETTE-ToS benchmark and the Multilingual Scraper Corpus show that full fine-tuning achieves the strongest precision recall balance, while LoRA-based models provide competitive recall with up to 3x lower memory cost. These findings highlight practical design trade-offs for efficient and domain-adapted LLMs, contributing open baselines for fine-tuning research in legal text processing.