TabSieve: Explicit In-Table Evidence Selection for Tabular Prediction

TL;DR

TabSieve is a framework that explicitly selects relevant table rows as evidence before predicting missing values, improving robustness and accuracy in tabular prediction tasks.

Contribution

It introduces a select-then-predict approach with a synthesized training dataset and reinforcement learning for joint evidence selection and prediction optimization.

Findings

Improves classification accuracy by 2.92% and regression by 4.45% over baselines.

Enhances robustness to noisy context by focusing on selected evidence.

Demonstrates consistent performance gains across various shot budgets.

Abstract

Tabular prediction can benefit from in-table rows as few-shot evidence, yet existing tabular models typically perform instance-wise inference and LLM-based prompting is often brittle. Models do not consistently leverage relevant rows, and noisy context can degrade performance. To address this challenge, we propose TabSieve, a select-then-predict framework that makes evidence usage explicit and auditable. Given a table and a query row, TabSieve first selects a small set of informative rows as evidence and then predicts the missing target conditioned on the selected evidence. To enable this capability, we construct TabSieve-SFT-40K by synthesizing high-quality reasoning trajectories from 331 real tables using a strong teacher model with strict filtering. Furthermore, we introduce TAB-GRPO, a reinforcement learning recipe that jointly optimizes evidence selection and prediction correctness with separate rewards, and stabilizes mixed regression and classification training via dynamic task-advantage balancing. Experiments on a held-out benchmark of 75 classification and 52 regression tables show that TabSieve consistently improves performance across shot budgets, with average gains of 2.92% on classification and 4.45% on regression over the second-best baseline. Further analysis indicates that TabSieve concentrates more attention on the selected evidence, which improves robustness to noisy context.