Neural Alignment for Face De-pixelization

TL;DR

This paper introduces a neural alignment method that reconstructs high-resolution, identifiable face videos from pixelated versions by leveraging frame similarity, spatial transformation, and adversarial learning, revealing privacy vulnerabilities.

Contribution

It presents a novel neural framework that aligns and reconstructs high-quality faces from pixelated videos without explicit temporal constraints, exposing privacy risks.

Findings

High-quality face reconstructions from pixelated videos are possible.

Alignment of neighboring frames enhances reconstruction accuracy.

The method reveals privacy vulnerabilities in face pixelation techniques.

Abstract

We present a simple method to reconstruct a high-resolution video from a face-video, where the identity of a person is obscured by pixelization. This concealment method is popular because the viewer can still perceive a human face figure and the overall head motion. However, we show in our experiments that a fairly good approximation of the original video can be reconstructed in a way that compromises anonymity. Our system exploits the simultaneous similarity and small disparity between close-by video frames depicting a human face, and employs a spatial transformation component that learns the alignment between the pixelated frames. Each frame, supported by its aligned surrounding frames, is first encoded, then decoded to a higher resolution. Reconstruction and perceptual losses promote adherence to the ground-truth, and an adversarial loss assists in maintaining domain faithfulness. There is no need for explicit temporal coherency loss as it is maintained implicitly by the alignment of neighboring frames and reconstruction. Although simple, our framework synthesizes high-quality face reconstructions, demonstrating that given the statistical prior of a human face, multiple aligned pixelated frames contain sufficient information to reconstruct a high-quality approximation of the original signal.