Generative Compression for Face Video: A Hybrid Scheme

TL;DR

This paper introduces a hybrid face video compression scheme combining traditional coding and deep learning, achieving high quality at ultra-low bitrates and outperforming VVC in fair comparisons.

Contribution

A novel hybrid compression scheme for face videos that integrates pixel-level recovery and deep learning, adaptable to various encoders and network conditions.

Findings

Achieves PSNR up to 36.23 dB at 1.47 KB/s

Outperforms VVC in fair comparison experiments

Can dynamically adjust bitrate based on network conditions

Abstract

As the latest video coding standard, versatile video coding (VVC) has shown its ability in retaining pixel quality. To excavate more compression potential for video conference scenarios under ultra-low bitrate, this paper proposes a bitrate adjustable hybrid compression scheme for face video. This hybrid scheme combines the pixel-level precise recovery capability of traditional coding with the generation capability of deep learning based on abridged information, where Pixel wise Bi-Prediction, Low-Bitrate-FOM and Lossless Keypoint Encoder collaborate to achieve PSNR up to 36.23 dB at a low bitrate of 1.47 KB/s. Without introducing any additional bitrate, our method has a clear advantage over VVC under a completely fair comparative experiment, which proves the effectiveness of our proposed scheme. Moreover, our scheme can adapt to any existing encoder / configuration to deal with different encoding requirements, and the bitrate can be dynamically adjusted according to the network condition.