Practical Wide-Angle Portraits Correction with Deep Structured Models

TL;DR

This paper presents a deep learning approach for correcting perspective distortions in wide-angle portraits, effectively removing artifacts and improving image quality without needing camera parameters.

Contribution

It introduces a cascaded deep network architecture and a new dataset for perspective portrait correction, outperforming previous methods.

Findings

Outperforms previous state-of-the-art methods quantitatively and qualitatively

Does not require camera distortion parameters

Introduces a new dataset and evaluation metrics

Abstract

Wide-angle portraits often enjoy expanded views. However, they contain perspective distortions, especially noticeable when capturing group portrait photos, where the background is skewed and faces are stretched. This paper introduces the first deep learning based approach to remove such artifacts from freely-shot photos. Specifically, given a wide-angle portrait as input, we build a cascaded network consisting of a LineNet, a ShapeNet, and a transition module (TM), which corrects perspective distortions on the background, adapts to the stereographic projection on facial regions, and achieves smooth transitions between these two projections, accordingly. To train our network, we build the first perspective portrait dataset with a large diversity in identities, scenes and camera modules. For the quantitative evaluation, we introduce two novel metrics, line consistency and face congruence. Compared to the previous state-of-the-art approach, our method does not require camera distortion parameters. We demonstrate that our approach significantly outperforms the previous state-of-the-art approach both qualitatively and quantitatively.