Mini-BEHAVIOR-Gran: Revealing U-Shaped Effects of Instruction Granularity on Language-Guided Embodied Agents

TL;DR

This paper introduces Mini-BEHAVIOR-Gran, a benchmark to study how different levels of instruction detail affect embodied AI performance, revealing a U-shaped relationship with optimal granularity at both extremes.

Contribution

It provides a new benchmark with multiple instruction variants per task and compares metrics for instruction granularity, uncovering a non-monotonic effect on agent performance.

Findings

Width correlates most with agent performance among metrics.

Performance shows a U-shaped relationship with instruction granularity.

Coarse instructions lead to vision-dominant policies.

Abstract

Instruction granularity is an important yet poorly controlled variable in language-guided embodied AI. Existing benchmarks typically pair each task with a single static instruction, making it difficult to study how agent behavior changes when the same task is described at different levels of detail. We introduce Mini-BEHAVIOR-Gran, a new benchmark for controlled studies of instruction granularity that extends Mini-BEHAVIOR with multiple instruction variants per task, ranging from high-level goal descriptions to step-by-step guidance. Using this benchmark, we compare four candidate metrics for cross-task granularity quantification: token count, entity count, action-verb count, and planning-width, and find that width correlates most consistently with agent performance. Using width to organize training and evaluation further reveals a non-monotonic U-shaped relationship between instruction granularity and performance, with peaks at both fine and coarse extremes. Further analysis suggests that the coarse-granularity performance rebound is associated with shallow grounding, where agents learn vision-dominant policies.