Task-agnostic Temporally Consistent Facial Video Editing

TL;DR

This paper introduces a unified, task-agnostic framework for facial video editing that ensures temporal consistency and photo-realism by leveraging 3D reconstruction and novel loss constraints.

Contribution

It presents a new framework that handles multiple facial editing tasks with improved temporal consistency and disentangled control, unlike prior task-specific methods.

Findings

Produces more photo-realistic video portraits

Achieves smoother temporal consistency in edits

Outperforms state-of-the-art facial editing methods

Abstract

Recent research has witnessed the advances in facial image editing tasks. For video editing, however, previous methods either simply apply transformations frame by frame or utilize multiple frames in a concatenated or iterative fashion, which leads to noticeable visual flickers. In addition, these methods are confined to dealing with one specific task at a time without any extensibility. In this paper, we propose a task-agnostic temporally consistent facial video editing framework. Based on a 3D reconstruction model, our framework is designed to handle several editing tasks in a more unified and disentangled manner. The core design includes a dynamic training sample selection mechanism and a novel 3D temporal loss constraint that fully exploits both image and video datasets and enforces temporal consistency. Compared with the state-of-the-art facial image editing methods, our framework generates video portraits that are more photo-realistic and temporally smooth.