Ontology-Aware RAG for Improved Question-Answering in Cybersecurity Education

TL;DR

This paper introduces CyberRAG, an ontology-aware retrieval-augmented generation system that improves the accuracy and reliability of AI-driven question-answering in cybersecurity education by integrating domain knowledge and validation mechanisms.

Contribution

It presents a novel CyberRAG framework combining knowledge retrieval and ontology validation to enhance AI reliability in cybersecurity education.

Findings

CyberRAG outperforms baseline models in accuracy and reliability.

The system effectively reduces hallucinations and misinformation.

Experimental results show improved alignment with domain knowledge.

Abstract

Integrating AI into education has the potential to transform the teaching of science and technology courses, particularly in the field of cybersecurity. AI-driven question-answering (QA) systems can actively manage uncertainty in cybersecurity problem-solving, offering interactive, inquiry-based learning experiences. Recently, Large language models (LLMs) have gained prominence in AI-driven QA systems, enabling advanced language understanding and user engagement. However, they face challenges like hallucinations and limited domain-specific knowledge, which reduce their reliability in educational settings. To address these challenges, we propose CyberRAG, an ontology-aware retrieval-augmented generation (RAG) approach for developing a reliable and safe QA system in cybersecurity education. CyberRAG employs a two-step approach: first, it augments the domain-specific knowledge by retrieving validated cybersecurity documents from a knowledge base to enhance the relevance and accuracy of the response. Second, it mitigates hallucinations and misuse by integrating a knowledge graph ontology to validate the final answer. Comprehensive experiments on publicly available datasets reveal that CyberRAG delivers accurate, reliable responses aligned with domain knowledge, demonstrating the potential of AI tools to enhance education.