DiffVC-OSD: One-Step Diffusion-based Perceptual Neural Video Compression Framework

TL;DR

DiffVC-OSD introduces a single-step diffusion-based neural video compression framework that significantly improves perceptual quality, reduces bitrate, and accelerates decoding compared to multi-step methods by leveraging temporal context and end-to-end finetuning.

Contribution

The paper presents a novel one-step diffusion model for neural video compression, incorporating a Temporal Context Adapter and end-to-end finetuning for enhanced performance.

Findings

Achieves state-of-the-art perceptual compression quality.

Decodes approximately 20 times faster than multi-step diffusion methods.

Reduces bitrate by 86.92% compared to multi-step diffusion variants.

Abstract

In this work, we first propose DiffVC-OSD, a One-Step Diffusion-based Perceptual Neural Video Compression framework. Unlike conventional multi-step diffusion-based methods, DiffVC-OSD feeds the reconstructed latent representation directly into a One-Step Diffusion Model, enhancing perceptual quality through a single diffusion step guided by both temporal context and the latent itself. To better leverage temporal dependencies, we design a Temporal Context Adapter that encodes conditional inputs into multi-level features, offering more fine-grained guidance for the Denoising Unet. Additionally, we employ an End-to-End Finetuning strategy to improve overall compression performance. Extensive experiments demonstrate that DiffVC-OSD achieves state-of-the-art perceptual compression performance, offers about 20$\times$ faster decoding and a 86.92\% bitrate reduction compared to the corresponding multi-step diffusion-based variant.