MRT at SemEval-2025 Task 8: Maximizing Recovery from Tables with Multiple Steps

TL;DR

This paper presents a multi-step approach using Python code generation with LLMs to answer questions over tables, involving understanding, instruction generation, code translation, execution, and error handling, achieving 70.50% score.

Contribution

It introduces a novel multi-step method leveraging LLMs and fine-tuned prompts for table question-answering, enhancing accuracy and robustness.

Findings

Achieved 70.50% score on subtask 1.

Demonstrated effectiveness of multi-step LLM-based approach.

Utilized open source LLMs with optimized prompts.

Abstract

In this paper we expose our approach to solve the \textit{SemEval 2025 Task 8: Question-Answering over Tabular Data} challenge. Our strategy leverages Python code generation with LLMs to interact with the table and get the answer to the questions. The process is composed of multiple steps: understanding the content of the table, generating natural language instructions in the form of steps to follow in order to get the answer, translating these instructions to code, running it and handling potential errors or exceptions. These steps use open source LLMs and fine grained optimized prompts for each task (step). With this approach, we achieved a score of $70.50\%$ for subtask 1.