# Lightning Fast Caching-based Parallel Denoising Prediction for Accelerating Talking Head Generation

## TL;DR

This paper introduces LightningCP, a caching-based parallel denoising method, and DFA, a decoupled attention mechanism, to significantly accelerate talking head video generation without sacrificing quality.

## Contribution

The paper presents a novel task-specific framework combining caching and decoupled attention to speed up diffusion-based talking head generation.

## Key findings

- Significant inference speedup demonstrated in experiments.
- High-quality videos maintained despite acceleration techniques.
- Effective exploitation of spatial and temporal redundancies in videos.

## Abstract

Diffusion-based talking head models generate high-quality, photorealistic videos but suffer from slow inference, limiting practical applications. Existing acceleration methods for general diffusion models fail to exploit the temporal and spatial redundancies unique to talking head generation. In this paper, we propose a task-specific framework addressing these inefficiencies through two key innovations. First, we introduce Lightning-fast Caching-based Parallel denoising prediction (LightningCP), caching static features to bypass most model layers in inference time. We also enable parallel prediction using cached features and estimated noisy latents as inputs, efficiently bypassing sequential sampling. Second, we propose Decoupled Foreground Attention (DFA) to further accelerate attention computations, exploiting the spatial decoupling in talking head videos to restrict attention to dynamic foreground regions. Additionally, we remove reference features in certain layers to bring extra speedup. Extensive experiments demonstrate that our framework significantly improves inference speed while preserving video quality.