A Lightweight Dual-Mode Optimization for Generative Face Video Coding

TL;DR

This paper introduces a lightweight dual-mode optimization framework for Generative Face Video Coding that significantly reduces model complexity and computational costs while maintaining high-quality reconstruction, enabling deployment on resource-constrained devices.

Contribution

The paper proposes a novel dual-mode optimization combining architectural redesign and adaptive channel pruning to create a lightweight GFVC framework with reduced parameters and computation.

Findings

Achieves 90.4% parameter reduction and 88.9% computation saving.

Outperforms state-of-the-art VVC in perceptual quality metrics.

Enables efficient GFVC deployment on mobile edge devices.

Abstract

Generative Face Video Coding (GFVC) achieves superior rate-distortion performance by leveraging the strong inference capabilities of deep generative models. However, its practical deployment is hindered by large model parameters and high computational costs. To address this, we propose a lightweight GFVC framework that introduces dual-mode optimization -- combining architectural redesign and operational refinement -- to reduce complexity whilst preserving reconstruction quality. Architecturally, we replace traditional 3 x 3 convolutions with slimmer and more efficient layers, reducing complexity without compromising feature expressiveness. Operationally, we develop a two-stage adaptive channel pruning strategy: (1) soft pruning during training identifies redundant channels via learnable thresholds, and (2) hard pruning permanently eliminates these channels post-training using a derived mask. This dual-phase approach ensures both training stability and inference efficiency. Experimental results demonstrate that the proposed lightweight dual-mode optimization for GFVC can achieve 90.4% parameter reduction and 88.9% computation saving compared to the baseline, whilst achieving superior performance compared to state-of-the-art video coding standard Versatile Video Coding (VVC) in terms of perceptual-level quality metrics. As such, the proposed method is expected to enable efficient GFVC deployment in resource-constrained environments such as mobile edge devices.