MagicTryOn: Harnessing Diffusion Transformer for Garment-Preserving Video Virtual Try-on

TL;DR

MagicTryOn is a novel diffusion-transformer framework that significantly improves garment detail preservation and temporal consistency in video virtual try-on, enabling realistic and stable garment synthesis across video frames.

Contribution

It introduces a garment-preserving strategy, a spatiotemporal RoPE, and a distillation method for real-time inference, advancing the state-of-the-art in garment-preserving VVT.

Findings

Outperforms existing methods in garment detail fidelity.

Achieves superior temporal stability and reduced jitter.

Enables real-time inference without loss of quality.

Abstract

Video Virtual Try-On (VVT) aims to synthesize garments that appear natural across consecutive video frames, capturing both their dynamics and interactions with human motion. Despite recent progress, existing VVT methods still suffer from inadequate garment fidelity and limited spatiotemporal consistency. The reasons are: (1) under-exploitation of garment information, with limited garment cues being injected, resulting in weaker fine-detail fidelity; and (2) a lack of spatiotemporal modeling, which hampers cross-frame identity consistency and causes temporal jitter and appearance drift. In this paper, we present MagicTryOn, a diffusion-transformer based framework for garment-preserving video virtual try-on. To preserve fine-grained garment details, we propose a fine-grained garment-preservation strategy that disentangles garment cues and injects these decomposed priors into the denoising process. To improve temporal garment consistency and suppress jitter, we introduce a garment-aware spatiotemporal rotary positional embedding (RoPE) that extends RoPE within full self-attention, using spatiotemporal relative positions to modulate garment tokens. We further impose a mask-aware loss during training to enhance fidelity within garment regions. Moreover, we adopt distribution-matching distillation to compress the sampling trajectory to four steps, enabling real-time inference without degrading garment fidelity. Extensive quantitative and qualitative experiments demonstrate that MagicTryOn outperforms existing methods, delivering superior garment-detail fidelity and temporal stability in unconstrained settings.