Knowledge Management for Automobile Failure Analysis Using Graph RAG

TL;DR

This paper develops an optimized Graph RAG system combining LLMs and knowledge graphs to improve automobile failure analysis, significantly enhancing the quality of generated failure-related explanations.

Contribution

It introduces an optimized Graph RAG pipeline tailored for existing knowledge graphs, improving failure analysis in automotive engineering.

Findings

ROUGE F1 score improved by 157.6% on average

Enhanced ability to generate failure-related explanations

Effective knowledge transfer for young engineers

Abstract

This paper presents a knowledge management system for automobile failure analysis using retrieval-augmented generation (RAG) with large language models (LLMs) and knowledge graphs (KGs). In the automotive industry, there is a growing demand for knowledge transfer of failure analysis from experienced engineers to young engineers. However, failure events are phenomena that occur in a chain reaction, making them difficult for beginners to analyze them. While knowledge graphs, which can describe semantic relationships and structure information is effective in representing failure events, due to their capability of representing the relationships between components, there is much information in KGs, so it is challenging for young engineers to extract and understand sub-graphs from the KG. On the other hand, there is increasing interest in the use of Graph RAG, a type of RAG that combines LLMs and KGs for knowledge management. However, when using the current Graph RAG framework with an existing knowledge graph for automobile failures, several issues arise because it is difficult to generate executable queries for a knowledge graph database which is not constructed by LLMs. To address this, we focused on optimizing the Graph RAG pipeline for existing knowledge graphs. Using an original Q&A dataset, the ROUGE F1 score of the sentences generated by the proposed method showed an average improvement of 157.6% compared to the current method. This highlights the effectiveness of the proposed method for automobile failure analysis.