Correcting Face Distortion in Wide-Angle Videos

TL;DR

This paper introduces a novel video warping algorithm that corrects facial distortions in wide-angle videos by applying local stereographic projection and mesh warping, improving visual quality and user preference.

Contribution

It proposes a new stereographic projection-based mesh warping method with temporal coherence constraints for correcting wide-angle video distortions.

Findings

83.9% user preference for the proposed method

Developed a wide-angle video dataset for evaluation

Effective correction of facial distortions in wide-angle videos

Abstract

Video blogs and selfies are popular social media formats, which are often captured by wide-angle cameras to show human subjects and expanded background. Unfortunately, due to perspective projection, faces near corners and edges exhibit apparent distortions that stretch and squish the facial features, resulting in poor video quality. In this work, we present a video warping algorithm to correct these distortions. Our key idea is to apply stereographic projection locally on the facial regions. We formulate a mesh warp problem using spatial-temporal energy minimization and minimize background deformation using a line-preservation term to maintain the straight edges in the background. To address temporal coherency, we constrain the temporal smoothness on the warping meshes and facial trajectories through the latent variables. For performance evaluation, we develop a wide-angle video dataset with a wide range of focal lengths. The user study shows that 83.9% of users prefer our algorithm over other alternatives based on perspective projection.