Can Tabular Foundation Models Guide Exploration in Robot Policy Learning?

TL;DR

This paper introduces TFM-S3, a hybrid local-global exploration method for robot policy learning that leverages a pretrained tabular foundation model to improve sample efficiency and convergence.

Contribution

The paper presents a novel hybrid exploration approach combining local updates with global search guided by a foundation model, enhancing efficiency in continuous control tasks.

Findings

TFM-S3 accelerates early convergence in benchmarks.

It improves final performance over TD3 and baselines.

Uses a pretrained foundation model for efficient screening.

Abstract

Policy optimization in high-dimensional continuous control for robotics remains a challenging problem. Predominant methods are inherently local and often require extensive tuning and carefully chosen initial guesses for good performance, whereas more global and less initialization-sensitive search methods typically incur high rollout costs. We propose TFM-S3, a tabular hybrid local-global method for improving global exploration in robot policy learning with limited rollout cost. We interleave high-frequency local updates with intermittent rounds of global search. In each search round, we construct a dynamically updated low-dimensional policy subspace via SVD and perform iterative surrogate-guided refinement within this space. A pretrained tabular foundation model predicts candidate returns from a small context set, enabling large-scale screening with limited rollout cost. Experiments on continuous control benchmarks show that TFM-S3 consistently accelerates early-stage convergence and improves final performance compared to TD3 and population-based baselines under an identical rollout budget. These results demonstrate that foundation models are a powerful new tool for creating sample-efficient policy learning methods for continuous control in robotics.