Face Hallucination via Split-Attention in Split-Attention Network

TL;DR

This paper introduces a novel neural network architecture that enhances face hallucination by separately modeling facial structure and texture, leading to more realistic high-resolution face images and improved recognition performance.

Contribution

The paper proposes the ESAG and SISN models that effectively fuse facial structure and texture features for improved face hallucination and recognition.

Findings

Significant improvement in face image clarity.

Enhanced face recognition accuracy.

Effective fusion of structure and texture features.

Abstract

Recently, convolutional neural networks (CNNs) have been widely employed to promote the face hallucination due to the ability to predict high-frequency details from a large number of samples. However, most of them fail to take into account the overall facial profile and fine texture details simultaneously, resulting in reduced naturalness and fidelity of the reconstructed face, and further impairing the performance of downstream tasks (e.g., face detection, facial recognition). To tackle this issue, we propose a novel external-internal split attention group (ESAG), which encompasses two paths responsible for facial structure information and facial texture details, respectively. By fusing the features from these two paths, the consistency of facial structure and the fidelity of facial details are strengthened at the same time. Then, we propose a split-attention in split-attention network (SISN) to reconstruct photorealistic high-resolution facial images by cascading several ESAGs. Experimental results on face hallucination and face recognition unveil that the proposed method not only significantly improves the clarity of hallucinated faces, but also encourages the subsequent face recognition performance substantially. Codes have been released at https://github.com/mdswyz/SISN-Face-Hallucination.