MiMoTable: A Multi-scale Spreadsheet Benchmark with Meta Operations for Table Reasoning
Zheng Li, Yang Du, Mao Zheng, Mingyang Song

TL;DR
MiMoTable is a new multi-scale spreadsheet benchmark with real-world tables and meta operations, designed to better evaluate LLMs' table reasoning capabilities across diverse and complex scenarios.
Contribution
The paper introduces MiMoTable, a comprehensive benchmark with real-world spreadsheets and a novel difficulty criterion based on meta operations, filling gaps in existing table reasoning benchmarks.
Findings
Claude-3.5-Sonnet achieves 77.4% accuracy on MiMoTable.
Performance drops as benchmark difficulty increases.
MiMoTable reveals room for improvement in LLM table reasoning.
Abstract
Extensive research has been conducted to explore the capability of Large Language Models (LLMs) for table reasoning and has significantly improved the performance on existing benchmarks. However, tables and user questions in real-world applications are more complex and diverse, presenting an unignorable gap compared to the existing benchmarks. To fill the gap, we propose a \textbf{M}ult\textbf{i}-scale spreadsheet benchmark with \textbf{M}eta \textbf{o}perations for \textbf{Table} reasoning, named as MiMoTable. Specifically, MiMoTable incorporates two key features. First, the tables in MiMoTable are all spreadsheets used in real-world scenarios, which cover seven domains and contain different types. Second, we define a new criterion with six categories of meta operations for measuring the difficulty of each question in MiMoTable, simultaneously as a new perspective for measuring the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Database Systems and Queries · Spreadsheets and End-User Computing · Data Quality and Management
