HIME: Efficient Headshot Image Super-Resolution with Multiple Exemplars

TL;DR

HIME is an efficient neural network that enhances low-resolution headshot images by leveraging multiple high-quality exemplars, effectively handling misalignments and improving detail reconstruction with lower computational costs.

Contribution

The paper introduces HIME, a novel end-to-end network that manages multiple exemplars without facial priors and incorporates a correlation loss for detailed feature reconstruction.

Findings

Outperforms previous exemplar-guided super-resolution methods in quality.

Reduces computational cost significantly.

Achieves superior qualitative and quantitative results.

Abstract

A promising direction for recovering the lost information in low-resolution headshot images is utilizing a set of high-resolution exemplars from the same identity. Complementary images in the reference set can improve the generated headshot quality across many different views and poses. However, it is challenging to make the best use of multiple exemplars: the quality and alignment of each exemplar cannot be guaranteed. Using low-quality and mismatched images as references will impair the output results. To overcome these issues, we propose an efficient Headshot Image Super-Resolution with Multiple Exemplars network (HIME) method. Compared with previous methods, our network can effectively handle the misalignment between the input and the reference without requiring facial priors and learn the aggregated reference set representation in an end-to-end manner. Furthermore, to reconstruct more detailed facial features, we propose a correlation loss that provides a rich representation of the local texture in a controllable spatial range. Experimental results demonstrate that the proposed framework not only has significantly fewer computation cost than recent exemplar-guided methods but also achieves better qualitative and quantitative performance.