Ultra-low bitrate video conferencing using deep image animation

TL;DR

This paper introduces a deep learning-based method for ultra-low bitrate video conferencing that encodes motion as keypoints, achieving over 80% bitrate reduction while maintaining visual quality.

Contribution

It presents a novel model-based deep neural network approach for ultra-low bitrate video compression tailored for video conferencing.

Findings

Achieves over 80% bitrate reduction compared to HEVC.

Maintains comparable visual quality at extremely low bitrates.

Demonstrates effectiveness through objective and subjective evaluations.

Abstract

In this work we propose a novel deep learning approach for ultra-low bitrate video compression for video conferencing applications. To address the shortcomings of current video compression paradigms when the available bandwidth is extremely limited, we adopt a model-based approach that employs deep neural networks to encode motion information as keypoint displacement and reconstruct the video signal at the decoder side. The overall system is trained in an end-to-end fashion minimizing a reconstruction error on the encoder output. Objective and subjective quality evaluation experiments demonstrate that the proposed approach provides an average bitrate reduction for the same visual quality of more than 80% compared to HEVC.