ALKAFI-LLAMA3: Fine-Tuning LLMs for Precise Legal Understanding in Palestine

TL;DR

This paper introduces ALKAFI-LLAMA3, a fine-tuned LLM tailored for Palestinian legal understanding, demonstrating effective, cost-efficient legal assistance in resource-limited settings through synthetic data training.

Contribution

It presents a novel fine-tuning approach using synthetic Palestinian legal data on a quantized Llama-3 model for low-resource legal AI applications.

Findings

Achieved accurate legal query responses across various question types.

Demonstrated cost-effective, locally sustainable model training.

Identified areas for future improvement like calculation handling.

Abstract

Large Language Models (LLMs) have demonstrated remarkable potential in diverse domains, yet their application in the legal sector, particularly in low-resource contexts, remains limited. This study addresses the challenges of adapting LLMs to the Palestinian legal domain, where political instability, fragmented legal frameworks, and limited AI resources hinder effective machine-learning applications. We present a fine-tuned model based on a quantized version of Llama-3.2-1B-Instruct, trained on a synthetic data set derived from Palestinian legal texts. Using smaller-scale models and strategically generated question-answer pairs, we achieve a cost-effective, locally sustainable solution that provides accurate and contextually relevant legal guidance. Our experiments demonstrate promising performance on various query types, ranging from yes/no questions and narrative explanations to complex legal differentiations, while highlighting areas for improvement, such as handling calculation-based inquiries and structured list formatting. This work provides a pathway for the deployment of AI-driven legal assistance tools tailored to the needs of resource-constrained environments.