Towards Robust Real-World Spreadsheet Understanding with Multi-Agent Multi-Format Reasoning

TL;DR

SpreadsheetAgent introduces a multi-agent, multi-modal framework that incrementally interprets and reasons over complex, large-scale spreadsheets, improving robustness and accuracy in real-world applications.

Contribution

The paper presents a novel two-stage multi-agent approach that leverages multiple modalities and verification to enhance spreadsheet understanding beyond existing LLM-based methods.

Findings

Achieves 38.16% on Spreadsheet Bench, surpassing baseline by 2.89 points.

Effectively interprets large, complex spreadsheets using incremental, localized reasoning.

Demonstrates robustness and scalability in real-world spreadsheet tasks.

Abstract

Spreadsheets are central to real-world applications such as enterprise reporting, auditing, and scientific data management. Despite their ubiquity, existing large language model based approaches typically treat tables as plain text, overlooking critical layout cues and visual semantics. Moreover, real-world spreadsheets are often massive in scale, exceeding the input length that LLMs can efficiently process. To address these challenges, we propose SpreadsheetAgent, a two-stage multi-agent framework for spreadsheet understanding that adopts a step-by-step reading and reasoning paradigm. Instead of loading the entire spreadsheet at once, SpreadsheetAgent incrementally interprets localized regions through multiple modalities, including code execution results, images, and LaTeX tables. The method first constructs a structural sketch and row/column summaries, and then performs task-driven reasoning over this intermediate representation in the Solving Stage. To further enhance reliability, we design a verification module that validates extracted structures via targeted inspections, reducing error propagation and ensuring trustworthy inputs for downstream reasoning. Extensive experiments on two spreadsheet datasets demonstrate the effectiveness of our approach. With GPT-OSS-120B, SpreadsheetAgent achieves 38.16% on Spreadsheet Bench, outperforming the ChatGPT Agent baseline (35.27%) by 2.89 absolute points. These results highlight the potential of SpreadsheetAgent to advance robust and scalable spreadsheet understanding in real-world applications. Code is available at https://github.com/renhouxing/SpreadsheetAgent.git.