Analysis and Benchmarking of Extending Blind Face Image Restoration to Videos

TL;DR

This paper introduces a benchmark for evaluating blind face restoration in videos, analyzes existing methods' limitations, and proposes a Temporal Consistency Network to improve video quality by reducing jitters and flickers.

Contribution

It presents a new benchmark for face video restoration, systematically analyzes challenges, and proposes a novel TCN method to enhance temporal consistency in restored videos.

Findings

The benchmark enables fair comparison of face video restoration methods.

Existing image-based algorithms face issues like jitters and flickers in videos.

The proposed TCN significantly reduces temporal artifacts in restored videos.

Abstract

Recent progress in blind face restoration has resulted in producing high-quality restored results for static images. However, efforts to extend these advancements to video scenarios have been minimal, partly because of the absence of benchmarks that allow for a comprehensive and fair comparison. In this work, we first present a fair evaluation benchmark, in which we first introduce a Real-world Low-Quality Face Video benchmark (RFV-LQ), evaluate several leading image-based face restoration algorithms, and conduct a thorough systematical analysis of the benefits and challenges associated with extending blind face image restoration algorithms to degraded face videos. Our analysis identifies several key issues, primarily categorized into two aspects: significant jitters in facial components and noise-shape flickering between frames. To address these issues, we propose a Temporal Consistency Network (TCN) cooperated with alignment smoothing to reduce jitters and flickers in restored videos. TCN is a flexible component that can be seamlessly plugged into the most advanced face image restoration algorithms, ensuring the quality of image-based restoration is maintained as closely as possible. Extensive experiments have been conducted to evaluate the effectiveness and efficiency of our proposed TCN and alignment smoothing operation. Project page: https://wzhouxiff.github.io/projects/FIR2FVR/FIR2FVR.