# Face Hallucination by Attentive Sequence Optimization with Reinforcement Learning

## TL;DR

This paper introduces a novel reinforcement learning-based face hallucination framework that sequentially attends to facial regions for improved super-resolution, outperforming existing patch-wise methods especially under challenging conditions.

## Contribution

The paper proposes a new attention-aware face hallucination framework using reinforcement learning to dynamically focus on facial regions for enhanced super-resolution.

## Key findings

- Outperforms state-of-the-art methods on in-the-wild face images
- Effectively handles large pose and illumination variations
- Utilizes global interdependency for better facial detail reconstruction

## Abstract

Face hallucination is a domain-specific super-resolution problem that aims to generate a high-resolution (HR) face image from a low-resolution~(LR) input. In contrast to the existing patch-wise super-resolution models that divide a face image into regular patches and independently apply LR to HR mapping to each patch, we implement deep reinforcement learning and develop a novel attention-aware face hallucination (Attention-FH) framework, which recurrently learns to attend a sequence of patches and performs facial part enhancement by fully exploiting the global interdependency of the image. Specifically, our proposed framework incorporates two components: a recurrent policy network for dynamically specifying a new attended region at each time step based on the status of the super-resolved image and the past attended region sequence, and a local enhancement network for selected patch hallucination and global state updating. The Attention-FH model jointly learns the recurrent policy network and local enhancement network through maximizing a long-term reward that reflects the hallucination result with respect to the whole HR image. Extensive experiments demonstrate that our Attention-FH significantly outperforms the state-of-the-art methods on in-the-wild face images with large pose and illumination variations.