A Graph-based RAG for Energy Efficiency Question Answering

TL;DR

This paper presents a graph-based Retrieval Augmented Generation system utilizing Large Language Models for energy efficiency question answering, demonstrating promising accuracy and multilingual capabilities through knowledge graph reasoning.

Contribution

The work introduces a novel graph-based RAG architecture for EE QA that automatically extracts and reasons over a knowledge graph, enhancing multilingual answer accuracy.

Findings

Achieves 75.2% overall correct answers.

Higher accuracy (81%) on general EE questions.

Maintains multilingual performance with only 4.4% translation accuracy loss.

Abstract

In this work, we investigate the use of Large Language Models (LLMs) within a graph-based Retrieval Augmented Generation (RAG) architecture for Energy Efficiency (EE) Question Answering. First, the system automatically extracts a Knowledge Graph (KG) from guidance and regulatory documents in the energy field. Then, the generated graph is navigated and reasoned upon to provide users with accurate answers in multiple languages. We implement a human-based validation using the RAGAs framework properties, a validation dataset comprising 101 question-answer pairs, and domain experts. Results confirm the potential of this architecture and identify its strengths and weaknesses. Validation results show how the system correctly answers in about three out of four of the cases (75.2 +- 2.7%), with higher results on questions related to more general EE answers (up to 81.0 +- 4.1%), and featuring promising multilingual abilities (4.4% accuracy loss due to translation).