From Table to Cell: Attention for Better Reasoning with TABALIGN

TL;DR

This paper introduces TABALIGN, a new framework for multi-step reasoning over tables using diffusion language models and cell attention verification, significantly improving accuracy and efficiency.

Contribution

It proposes a planned reasoning framework with a bidirectional DLM planner and a cell attention verifier, enhancing permutation invariance and reasoning accuracy.

Findings

DLMs produce more human-aligned, permutation-stable cell attention than autoregressive models.

TABALIGN improves accuracy by 15.76 percentage points over strong open-source baselines.

Cleaner DLM plans accelerate reasoning execution by 44.64%.

Abstract

Multi-step LLM reasoning over structured tables fails because planning and execution share no explicit cell-grounding contract. Existing methods constrain the planner to a left-to-right factorization at odds with table permutation invariance, and score intermediate states by generated content alone, overlooking cell grounding. We conduct a pilot study showing that diffusion language models (DLMs) produce more human-aligned and permutation-stable cell attention on tables than autoregressive models, with a 40.2% median reduction in attention-AUROC variability under row reordering. Motivated by this, we propose TABALIGN, a planned table reasoning framework that operationalizes the contract. TABALIGN pairs a masked DLM planner, whose bidirectional denoising emits plan steps as binary cell masks, with TABATTN, a lightweight verifier trained on 1,600 human-verified attention standards to score each step by its attention overlap with the plan-designated mask. Across eight benchmarks covering table question answering and fact verification, TABALIGN improves average accuracy by 15.76 percentage points over the strongest open-source baseline at comparable 8B-class scale, with a matched-backbone ablation attributing 2.87 percentage points of this gain to the DLM planner over an AR planner on a fixed reasoner. Cleaner DLM plans also accelerate downstream reasoning execution by 44.64%.