Hybrid Retrieval-Augmented Generation Agent for Trustworthy Legal Question Answering in Judicial Forensics

TL;DR

This paper introduces a hybrid legal question answering system that combines retrieval-augmented generation with multi-model ensembling and human review to improve answer reliability, traceability, and updateability in judicial forensics.

Contribution

It presents a novel hybrid QA framework integrating retrieval, multi-model ensembling, and human-in-the-loop updates for trustworthy legal answers.

Findings

Outperforms baseline models on Law_QA dataset in F1, ROUGE-L, and LLM-as-a-Judge metrics.

Reduces hallucination and improves answer quality in legal QA.

Enables dynamic knowledge updates with provenance tracking.

Abstract

As artificial intelligence permeates judicial forensics, ensuring the veracity and traceability of legal question answering (QA) has become critical. Conventional large language models (LLMs) are prone to hallucination, risking misleading guidance in legal consultation, while static knowledge bases struggle to keep pace with frequently updated statutes and case law. We present a hybrid legal QA agent tailored for judicial settings that integrates retrieval-augmented generation (RAG) with multi-model ensembling to deliver reliable, auditable, and continuously updatable counsel. The system prioritizes retrieval over generation: when a trusted legal repository yields relevant evidence, answers are produced via RAG; otherwise, multiple LLMs generate candidates that are scored by a specialized selector, with the top-ranked answer returned. High-quality outputs then undergo human review before being written back to the repository, enabling dynamic knowledge evolution and provenance tracking. Experiments on the Law\_QA dataset show that our hybrid approach significantly outperforms both a single-model baseline and a vanilla RAG pipeline on F1, ROUGE-L, and an LLM-as-a-Judge metric. Ablations confirm the complementary contributions of retrieval prioritization, model ensembling, and the human-in-the-loop update mechanism. The proposed system demonstrably reduces hallucination while improving answer quality and legal compliance, advancing the practical landing of media forensics technologies in judicial scenarios.