Value Vision-Language-Action Planning & Search

TL;DR

This paper introduces V-VLAPS, a framework that enhances vision-language-action planning with a learnable value function, significantly improving robotic manipulation success rates and efficiency over prior methods that rely solely on priors.

Contribution

The paper proposes V-VLAPS, integrating a lightweight value function into MCTS for VLA models, providing explicit success signals to improve planning performance.

Findings

V-VLAPS increases success rates by over 5 percentage points.

Reduces MCTS simulations by 5-15% compared to prior methods.

Demonstrates effectiveness on the LIBERO robotic manipulation suite.

Abstract

Vision-Language-Action (VLA) models have emerged as powerful generalist policies for robotic manipulation, yet they remain fundamentally limited by their reliance on behavior cloning, leading to brittleness under distribution shift. While augmenting pretrained models with test-time search algorithms like Monte Carlo Tree Search (MCTS) can mitigate these failures, existing formulations rely solely on the VLA prior for guidance, lacking a grounded estimate of expected future return. Consequently, when the prior is inaccurate, the planner can only correct action selection via the exploration term, which requires extensive simulation to become effective. To address this limitation, we introduce Value Vision-Language-Action Planning and Search (V-VLAPS), a framework that augments MCTS with a lightweight, learnable value function. By training a simple multilayer perceptron (MLP) on the latent representations of a fixed VLA backbone (Octo), we provide the search with an explicit success signal that biases action selection toward high-value regions. We evaluate V-VLAPS on the LIBERO robotic manipulation suite, demonstrating that our value-guided search improves success rates by over 5 percentage points while reducing the average number of MCTS simulations by 5-15 percent compared to baselines that rely only on the VLA prior.