TTQA-RS- A break-down prompting approach for Multi-hop Table-Text Question Answering with Reasoning and Summarization

TL;DR

This paper introduces TTQA-RS, a retrieval-augmented, break-down prompting approach for multi-hop table-text question answering that leverages enhanced retrieval and reasoning to outperform existing methods on benchmark datasets.

Contribution

The paper presents a novel retrieval-augmented generation model with a break-down prompting strategy for multi-hop table-text QA, achieving state-of-the-art results with open-source LLMs.

Findings

Outperforms existing prompting methods on HybridQA and OTT-QA datasets.

Uses an enhanced retriever for better table-text information retrieval.

Achieves state-of-the-art performance with LLaMA3-70B.

Abstract

Question answering (QA) over tables and text has gained much popularity over the years. Multi-hop table-text QA requires multiple hops between the table and text, making it a challenging QA task. Although several works have attempted to solve the table-text QA task, most involve training the models and requiring labeled data. In this paper, we have proposed a Retrieval Augmented Generation (RAG) based model - TTQA-RS: A break-down prompting approach for Multi-hop Table-Text Question Answering with Reasoning and Summarization. Our model uses an enhanced retriever for table-text information retrieval and uses augmented knowledge, including table-text summary with decomposed sub-questions with answers for a reasoning-based table-text QA. Using open-source language models, our model outperformed all existing prompting methods for table-text QA tasks on existing table-text QA datasets, such as HybridQA and OTT-QA's development set. Our experiments demonstrate the potential of prompt-based approaches using open-source LLMs. Additionally, by using LLaMA3-70B, our model achieved state-of-the-art performance for prompting-based methods on multi-hop table-text QA.