Knowledge in Triples for LLMs: Enhancing Table QA Accuracy with Semantic Extraction

TL;DR

This paper introduces a method that extracts triples from complex tables and uses retrieval-augmented generation to improve the accuracy and detail of table-based question answering with GPT-3.5, outperforming existing methods.

Contribution

It presents a novel triple extraction technique combined with RAG to enhance table QA accuracy, especially on semi-structured web tables, surpassing traditional semantic approaches.

Findings

Outperforms baselines on FeTaQA dataset

Achieves higher Sacre-BLEU and ROUGE scores

Generates detailed, contextually accurate answers

Abstract

Integrating structured knowledge from tabular formats poses significant challenges within natural language processing (NLP), mainly when dealing with complex, semi-structured tables like those found in the FeTaQA dataset. These tables require advanced methods to interpret and generate meaningful responses accurately. Traditional approaches, such as SQL and SPARQL, often fail to fully capture the semantics of such data, especially in the presence of irregular table structures like web tables. This paper addresses these challenges by proposing a novel approach that extracts triples straightforward from tabular data and integrates it with a retrieval-augmented generation (RAG) model to enhance the accuracy, coherence, and contextual richness of responses generated by a fine-tuned GPT-3.5-turbo-0125 model. Our approach significantly outperforms existing baselines on the FeTaQA dataset, particularly excelling in Sacre-BLEU and ROUGE metrics. It effectively generates contextually accurate and detailed long-form answers from tables, showcasing its strength in complex data interpretation.