PAKTON: A Multi-Agent Framework for Question Answering in Long Legal Agreements

TL;DR

PAKTON is an open-source multi-agent framework that enhances legal contract review by combining collaborative workflows with retrieval-augmented generation, improving accuracy, explainability, and privacy in automated legal analysis.

Contribution

It introduces a novel multi-agent system with plug-and-play capabilities and a retrieval-augmented generation component for improved legal document review.

Findings

Outperforms general-purpose models in accuracy and retrieval

Enhances explainability and grounded justifications

Validated through human study and automated metrics

Abstract

Contract review is a complex and time-intensive task that typically demands specialized legal expertise, rendering it largely inaccessible to non-experts. Moreover, legal interpretation is rarely straightforward-ambiguity is pervasive, and judgments often hinge on subjective assessments. Compounding these challenges, contracts are usually confidential, restricting their use with proprietary models and necessitating reliance on open-source alternatives. To address these challenges, we introduce PAKTON: a fully open-source, end-to-end, multi-agent framework with plug-and-play capabilities. PAKTON is designed to handle the complexities of contract analysis through collaborative agent workflows and a novel retrieval-augmented generation (RAG) component, enabling automated legal document review that is more accessible, adaptable, and privacy-preserving. Experiments demonstrate that PAKTON outperforms both general-purpose and pretrained models in predictive accuracy, retrieval performance, explainability, completeness, and grounded justifications as evaluated through a human study and validated with automated metrics.