LLM-Augmented Knowledge Base Construction For Root Cause Analysis

TL;DR

This paper evaluates three LLM-based methods for building a knowledge base from support tickets to enhance root cause analysis in communication networks, aiming to improve outage response times.

Contribution

It introduces and compares Fine-Tuning, RAG, and Hybrid LLM approaches for RCA knowledge base construction from real industrial data.

Findings

The Hybrid approach outperforms others in similarity metrics.

The generated knowledge base accelerates RCA tasks.

Experiments demonstrate improved network resilience.

Abstract

Communications networks now form the backbone of our digital world, with fast and reliable connectivity. However, even with appropriate redundancy and failover mechanisms, it is difficult to guarantee "five 9s" (99.999 %) reliability, requiring rapid and accurate root cause analysis (RCA) during outages. In the event of an outage, rapid and accurate RCA becomes essential to restore service and prevent future disruptions. This study evaluates three Large Language Model (LLM) methodologies - Fine-Tuning, RAG, and a Hybrid approach - for constructing a Root Cause Analysis (RCA) Knowledge Base from support tickets. We compare their performance using a comprehensive suite of lexical and semantic similarity metrics. Our experiments on a real industrial dataset demonstrate that the generated knowledge base provides an excellent starting point for accelerating RCA tasks and improving network resilience.