The Observability Gap: Why Output-Level Human Feedback Fails for LLM Coding Agents

TL;DR

This paper investigates the limitations of output-only human feedback in guiding LLM-based coding agents, revealing a structural observability gap that hampers effective learning and proposing the need for intermediate feedback mechanisms.

Contribution

The study identifies a fundamental observability gap in output-only feedback for LLM coding agents and demonstrates that adding minimal code-level information can restore effective learning.

Findings

Output-only feedback fails to guide agents to full success in complex tasks.

A structural observability gap causes persistent failure modes.

Adding minimal code-level feedback restores convergence.

Abstract

Large language model (LLM) multi-agent coding systems typically fix agent capabilities at design time. We study an alternative setting, earned autonomy, in which a coding agent starts with zero pre-defined functions and incrementally builds a reusable function library through lightweight human feedback on visual output alone. We evaluate this setup in a Blender-based 3D scene generation task requiring both spatial reasoning and programmatic geometric control. Although the agent rediscovered core utility functions comparable to a human reference implementation, it achieved 0% full-scene success under output-only feedback across multiple instruction granularities, where success required satisfying object completeness, ground contact, collision avoidance, and scale plausibility simultaneously. Our analysis identifies a structural observability gap: bugs originate in code logic and execution state, while human evaluation occurs only at the output layer, and the many-to-one mapping from internal states to visible outcomes prevents symptom-level feedback from reliably identifying root causes. This mismatch leads to persistent failure mode oscillation rather than convergence. A diagnostic intervention that injected minimal code-level knowledge restored convergence, strongly supporting the interpretation that the main bottleneck lies in feedback observability rather than programming competence. We formalize this phenomenon as a feedback paradox in domains with deep causal chains between internal code logic and perceptual outcomes, and argue that effective human-agent collaboration in such settings requires intermediate observability beyond output-only evaluation.