Motion deblurring of faces

TL;DR

This paper introduces a data-driven face motion deblurring method that preserves identity by using dual sub-networks and a new dataset, improving face analysis tasks like recognition and landmark localization.

Contribution

The paper proposes a novel dual-stream neural network for face deblurring that maintains identity information and introduces a large realistic motion blur dataset from facial videos.

Findings

Outperforms existing deblurring methods in preserving face identity

Enhances accuracy in face recognition and landmark localization after deblurring

Demonstrates the effectiveness of deblurring as a preprocessing step for face analysis

Abstract

Face analysis is a core part of computer vision, in which remarkable progress has been observed in the past decades. Current methods achieve recognition and tracking with invariance to fundamental modes of variation such as illumination, 3D pose, expressions. Notwithstanding, a much less standing mode of variation is motion deblurring, which however presents substantial challenges in face analysis. Recent approaches either make oversimplifying assumptions, e.g. in cases of joint optimization with other tasks, or fail to preserve the highly structured shape/identity information. Therefore, we propose a data-driven method that encourages identity preservation. The proposed model includes two parallel streams (sub-networks): the first deblurs the image, the second implicitly extracts and projects the identity of both the sharp and the blurred image in similar subspaces. We devise a method for creating realistic motion blur by averaging a variable number of frames to train our model. The averaged images originate from a 2MF2 dataset with 10 million facial frames, which we introduce for the task. Considering deblurring as an intermediate step, we utilize the deblurred outputs to conduct a thorough experimentation on high-level face analysis tasks, i.e. landmark localization and face verification. The experimental evaluation demonstrates the superiority of our method.