Event-Grounded Sparse Autoencoders for Vision-Language-Action Policies

TL;DR

This paper introduces an event-grounded interpretability pipeline for vision-language-action policies, linking sparse autoencoder features to behavioral events to improve understanding and causal analysis of robot actions.

Contribution

It proposes a novel event-grounded analysis method for SAE features in VLA policies, enhancing causal interpretability and transferability across architectures and real robots.

Findings

Event-grounded ranking yields strong causal effects on OpenVLA.

The approach transfers to continuous action chunks of π_{0.5}.

Usability varies with architecture and intervention site, revealing safety and interpretability limits.

Abstract

Vision-Language-Action (VLA) policies translate language and visual inputs into robot actions, where their hidden representations directly shape closed-loop behavior. However, mechanistic interpretability tools from language and vision-language models do not transfer cleanly to VLAs: outputs are robot actions rather than human-readable tokens, and interventions can only be tested via expensive closed-loop rollouts. We propose an event-grounded interpretability pipeline that anchors SAE feature analysis to behavioral events rather than text contexts. End-effector keyframes are clustered within each task using visual, state, and temporal cues, linking SAE features to behaviorally salient events and, via optional VLM annotations, to semantic context. To our knowledge, our pipeline is among the first to ground SAE-based VLA analysis in closed-loop behavioral events. Across two simulation architectures and a real-robot study, event-grounded ranking yields the strongest causal effects on OpenVLA and transfers to the continuous action chunks of $\pi_{0.5}$. SAE is a sparse but imperfect intervention basis: usability varies with architecture and intervention site, and aggressive intervention reveals safety and interpretability limits. Overall, event-grounded SAE analysis emerges as a practical starting point for behavior-anchored VLA interpretability, motivating future work on SAE features beyond action-aligned coordinates, finer-grained closed-loop evaluation, and safe interventions for high-stakes VLA deployments. Code is available at \url{https://github.com/xc-j/Event-SAE}.