FlexCache: Flexible Approximate Cache System for Video Diffusion

TL;DR

FlexCache is a novel cache system for video diffusion that compresses caches and decouples objects from backgrounds, significantly improving throughput and reducing costs in text-to-video generation.

Contribution

We introduce FlexCache, a flexible approximate cache system with cache compression and object-background decoupling, tailored for efficient video diffusion.

Findings

Achieves 1.26x higher throughput

Reduces cache storage by 6.7x on average

Cuts costs by 25% compared to existing systems

Abstract

Text-to-Video applications receive increasing attention from the public. Among these, diffusion models have emerged as the most prominent approach, offering impressive quality in visual content generation. However, it still suffers from substantial computational complexity, often requiring several minutes to generate a single video. While prior research has addressed the computational overhead in text-to-image diffusion models, the techniques developed are not directly suitable for video diffusion models due to the significantly larger cache requirements and enhanced computational demands associated with video generation. We present FlexCache, a flexible approximate cache system that addresses the challenges in two main designs. First, we compress the caches before saving them to storage. Our compression strategy can reduce 6.7 times consumption on average. Then we find that the approximate cache system can achieve higher hit rate and computation savings by decoupling the object and background. We further design a tailored cache replacement policy to support the two techniques mentioned above better. Through our evaluation, FlexCache reaches 1.26 times higher throughput and 25% lower cost compared to the state-of-the-art diffusion approximate cache system.