Characterizing VLA Models: Identifying the Action Generation Bottleneck for Edge AI Architectures

TL;DR

This paper analyzes the performance bottlenecks of VLA models on edge hardware, identifying memory-bound action generation as a key latency contributor and exploring hardware solutions for scaling.

Contribution

It characterizes VLA model performance on edge devices, identifies the action-generation bottleneck, and evaluates hardware strategies for scaling to larger models.

Findings

Memory-bound action generation consumes up to 75% of latency.

High-bandwidth memory and PIM can mitigate bottlenecks.

Projected hardware needs for 100B parameter models.

Abstract

Vision-Language-Action (VLA) models are an emerging class of workloads critical for robotics and embodied AI at the edge. As these models scale, they demonstrate significant capability gains, yet they must be deployed locally to meet the strict latency requirements of real-time applications. This paper characterizes VLA performance on two generations of edge hardware, viz. the Nvidia Jetson Orin and Thor platforms. Using MolmoAct-7B, a state-of-the-art VLA model, we identify a primary execution bottleneck: up to 75% of end-to-end latency is consumed by the memory-bound action-generation phase. Through analytical modeling and simulations, we project the hardware requirements for scaling to 100B parameter models. We also explore the impact of high-bandwidth memory technologies and processing-in-memory (PIM) as promising future pathways in edge systems for embodied AI.