Retrieval Augmented Generation-based Large Language Models for Bridging Transportation Cybersecurity Legal Knowledge Gaps

TL;DR

This paper presents a Retrieval-Augmented Generation framework for large language models to improve legal knowledge retrieval and response accuracy in transportation cybersecurity legislation, outperforming existing models in factual grounding.

Contribution

Introduces a novel RAG-based LLM framework tailored for legal domain, reducing hallucinations and enhancing factual accuracy in legislative analysis for transportation cybersecurity.

Findings

Outperforms commercial LLMs on four evaluation metrics

Enhances factual grounding and specificity of legal responses

Supports scalable legislative analysis in transportation cybersecurity

Abstract

As connected and automated transportation systems evolve, there is a growing need for federal and state authorities to revise existing laws and develop new statutes to address emerging cybersecurity and data privacy challenges. This study introduces a Retrieval-Augmented Generation (RAG) based Large Language Model (LLM) framework designed to support policymakers by extracting relevant legal content and generating accurate, inquiry-specific responses. The framework focuses on reducing hallucinations in LLMs by using a curated set of domain-specific questions to guide response generation. By incorporating retrieval mechanisms, the system enhances the factual grounding and specificity of its outputs. Our analysis shows that the proposed RAG-based LLM outperforms leading commercial LLMs across four evaluation metrics: AlignScore, ParaScore, BERTScore, and ROUGE, demonstrating its effectiveness in producing reliable and context-aware legal insights. This approach offers a scalable, AI-driven method for legislative analysis, supporting efforts to update legal frameworks in line with advancements in transportation technologies.