Lightweight Domain Adaptation of a Large Language Model for Legal Assistance in the Indian Context

TL;DR

This paper presents Legal Assist AI, a highly efficient legal assistance framework for India using a small, 8-billion-parameter Llama model combined with retrieval-augmented generation and a comprehensive legal corpus, achieving high accuracy and efficiency.

Contribution

The paper introduces a novel, domain-specific legal assistance framework using a smaller, efficient model with retrieval augmentation, outperforming larger models in the Indian legal context.

Findings

Achieved 60.08% in the All-India Bar Examination, surpassing GPT-3.5 Turbo's 58.72%.

Demonstrated the 8B model is 22 times more parameter-efficient than a 175B baseline.

Successfully mitigated hallucinations in legal AI responses.

Abstract

In India, access to legal assistance for the general public has been observed to have a critical gap, as many citizens are not able to take full advantage of their legal rights due to limited access and awareness of apposite legal information. This paper thus introduces Legal Assist AI, a highly efficient framework designed to provide legal assistance in the Indian domain. The core contribution is a framework demonstrating how a smaller, 8-billion-parameter quantized model (Llama 3.1) can achieve superior domain-specific performance. This effective performance stems from integrating a Retrieval-Augmented Generation (RAG) system with strategic prompt engineering, supported by a high-quality, up to date corpus of more than 600 legal documents. This corpus includes the Indian Constitution and more importantly, the newly enacted Bharatiya Nyaya Sanhita (BNS) and Bharatiya Nagarik Suraksha Sanhita (BNSS) among others. Further, by achieving a score of 60.08\% in the All-India Bar Examination (AIBE) benchmark, the specialized approach based on RAG was found to be highly efficient and effective, improving on the 58.72\% score of the 175-billion parameter GPT-3.5 Turbo. It was also observed that the framework was able to manage and mitigate instances of hallucinations successfully, which is a critical requirement for practical legal applications. A Parameter Efficiency Index (PEI) is also introduced, with the goal of quantifying the superior efficiency that the framework was able to achieve, demonstrating how the 8B model is 22 times more parameter-efficient than the 175B baseline, and hence corroborating the potential of smaller domain-adapted models.