MobileI2V: Fast and High-Resolution Image-to-Video on Mobile Devices

TL;DR

MobileI2V introduces a lightweight diffusion model optimized for real-time, high-resolution image-to-video generation on mobile devices, achieving significant speed improvements while maintaining quality.

Contribution

The paper presents a novel lightweight diffusion model with a hybrid attention architecture and a time-step distillation strategy tailored for mobile devices, enabling fast 720p video generation.

Findings

Real-time 720p video generation on mobile devices.

10-fold speed-up with minimal quality loss.

Achieved frame generation under 100 ms.

Abstract

Recently, video generation has witnessed rapid advancements, drawing increasing attention to image-to-video (I2V) synthesis on mobile devices. However, the substantial computational complexity and slow generation speed of diffusion models pose significant challenges for real-time, high-resolution video generation on resource-constrained mobile devices. In this work, we propose MobileI2V, a 270M lightweight diffusion model for real-time image-to-video generation on mobile devices. The core lies in: (1) We analyzed the performance of linear attention modules and softmax attention modules on mobile devices, and proposed a linear hybrid architecture denoiser that balances generation efficiency and quality. (2) We design a time-step distillation strategy that compresses the I2V sampling steps from more than 20 to only two without significant quality loss, resulting in a 10-fold increase in generation speed. (3) We apply mobile-specific attention optimizations that yield a 2-fold speed-up for attention operations during on-device inference. MobileI2V enables, for the first time, fast 720p image-to-video generation on mobile devices, with quality comparable to existing models. Under one-step conditions, the generation speed of each frame of 720p video is less than 100 ms. Our code is available at: https://github.com/hustvl/MobileI2V.