Building Trust in the Skies: A Knowledge-Grounded LLM-based Framework for Aviation Safety

TL;DR

This paper presents a framework combining Large Language Models and Knowledge Graphs to improve trustworthiness and accuracy in aviation safety decision-making, addressing LLM limitations like hallucination.

Contribution

It introduces an end-to-end system that constructs and updates an Aviation Safety Knowledge Graph and uses it within a retrieval-augmented architecture to enhance LLM reliability.

Findings

Improved accuracy and traceability over LLM-only approaches

Enhanced support for complex safety queries

Effective mitigation of hallucination in safety analytics

Abstract

The integration of Large Language Models (LLMs) into aviation safety decision-making represents a significant technological advancement, yet their standalone application poses critical risks due to inherent limitations such as factual inaccuracies, hallucination, and lack of verifiability. These challenges undermine the reliability required for safety-critical environments where errors can have catastrophic consequences. To address these challenges, this paper proposes a novel, end-to-end framework that synergistically combines LLMs and Knowledge Graphs (KGs) to enhance the trustworthiness of safety analytics. The framework introduces a dual-phase pipeline: it first employs LLMs to automate the construction and dynamic updating of an Aviation Safety Knowledge Graph (ASKG) from multimodal sources. It then leverages this curated KG within a Retrieval-Augmented Generation (RAG) architecture to ground, validate, and explain LLM-generated responses. The implemented system demonstrates improved accuracy and traceability over LLM-only approaches, effectively supporting complex querying and mitigating hallucination. Results confirm the framework's capability to deliver context-aware, verifiable safety insights, addressing the stringent reliability requirements of the aviation industry. Future work will focus on enhancing relationship extraction and integrating hybrid retrieval mechanisms.