Efficient Multi-Agent Collaboration with Tool Use for Online Planning in Complex Table Question Answering

TL;DR

This paper introduces MACT, a multi-agent framework that uses tool-assisted collaboration for complex table question answering, achieving competitive results without fine-tuning or closed-source models.

Contribution

The paper presents MACT, a novel multi-agent approach that enables effective TQA without fine-tuning or relying on closed-source models, improving accessibility and reproducibility.

Findings

Outperforms previous state-of-the-art on three TQA benchmarks

Performs comparably to GPT-4 on two benchmarks using only open models

Extensive analysis confirms the effectiveness of multi-agent collaboration in TQA

Abstract

Complex table question answering (TQA) aims to answer questions that require complex reasoning, such as multi-step or multi-category reasoning, over data represented in tabular form. Previous approaches demonstrated notable performance by leveraging either closed-source large language models (LLMs) or fine-tuned open-weight LLMs. However, fine-tuning LLMs requires high-quality training data, which is costly to obtain, and utilizing closed-source LLMs poses accessibility challenges and leads to reproducibility issues. In this paper, we propose Multi-Agent Collaboration with Tool use (MACT), a framework that requires neither closed-source models nor fine-tuning. In MACT, a planning agent and a coding agent that also make use of tools collaborate to answer questions. Our experiments on four TQA benchmarks show that MACT outperforms previous SoTA systems on three out of four benchmarks and that it performs comparably to the larger and more expensive closed-source model GPT-4 on two benchmarks, even when using only open-weight models without any fine-tuning. We conduct extensive analyses to prove the effectiveness of MACT's multi-agent collaboration in TQA.