Table-r1: Self-supervised and Reinforcement Learning for Program-based Table Reasoning in Small Language Models

TL;DR

This paper introduces Table-r1, a novel two-stage self-supervised and reinforcement learning approach that significantly improves program-based table reasoning in small language models, narrowing the performance gap with larger models.

Contribution

The paper proposes a new two-stage method, combining self-supervised layout inference and reinforcement learning, to enhance small language models' ability to perform reliable program-based table reasoning.

Findings

Table-r1 improves SLM accuracy by at least 15% over baseline models.

The method achieves performance comparable to large language models on multiple benchmarks.

It effectively generalizes across diverse table layouts and reasoning tasks.

Abstract

Table reasoning (TR) requires structured reasoning over semi-structured tabular data and remains challenging, particularly for small language models (SLMs, e.g., LLaMA-8B) due to their limited capacity compared to large LMs (LLMs, e.g., GPT-4o). To narrow this gap, we explore program-based TR (P-TR), which circumvents key limitations of text-based TR (T-TR), notably in numerical reasoning, by generating executable programs. However, applying P-TR to SLMs introduces two challenges: (i) vulnerability to heterogeneity in table layouts, and (ii) inconsistency in reasoning due to limited code generation capability. We propose Table-r1, a two-stage P-TR method designed for SLMs. Stage 1 introduces an innovative self-supervised learning task, Layout Transformation Inference, to improve tabular layout generalization from a programmatic view. Stage 2 adopts a mix-paradigm variant of Group Relative Policy Optimization, enhancing P-TR consistency while allowing dynamic fallback to T-TR when needed. Experiments on four TR benchmarks demonstrate that Table-r1 outperforms all SLM-based methods, achieving at least a 15% accuracy improvement over the base model (LLaMA-8B) across all datasets and reaching performance competitive with LLMs.