FastSTAR: Spatiotemporal Token Pruning for Efficient Autoregressive Video Synthesis

TL;DR

FastSTAR introduces a spatiotemporal token pruning method that accelerates autoregressive video synthesis by selectively refining only essential regions, significantly reducing computation while maintaining high quality.

Contribution

It presents a training-free token pruning framework for STAR video generation, addressing the token explosion problem with a novel partial update mechanism.

Findings

Achieves up to 2.01x speedup in video synthesis

Maintains high PSNR of 28.29 with minimal quality loss

Demonstrates superior efficiency-quality trade-off on InfinityStar

Abstract

Visual Autoregressive modeling (VAR) has emerged as a highly efficient alternative to diffusion-based frameworks, achieving comparable synthesis quality. However, as this paradigm extends to Spacetime Autoregressive modeling (STAR) for video generation, scaling resolution and frame counts leads to a "token explosion" that creates a massive computational bottleneck in the final refinement stages. To address this, we propose FastSTAR, a training-free acceleration framework designed for high-quality video generation. Our core method, Spatiotemporal Token Pruning, identifies essential tokens by integrating two specialized terms: (1) Spatial similarity, which evaluates structural convergence across hierarchical scales to skip computations in regions where further refinement becomes redundant, and (2) Temporal similarity, which identifies active motion trajectories by assessing feature-level variations relative to the preceding clip. Combined with a Partial Update mechanism, FastSTAR ensures that only non-converged regions are refined, maintaining fluid motion while bypassing redundant computations. Experimental results on InfinityStar demonstrate that FastSTAR achieves up to a 2.01x speedup with a PSNR of 28.29 and less than 1% performance degradation, proving a superior efficiency-quality trade-off for STAR-based video synthesis.