Face2Scene: Using Facial Degradation as an Oracle for Diffusion-Based Scene Restoration

TL;DR

Face2Scene introduces a novel two-stage scene restoration method that uses facial cues as an oracle to guide the restoration of entire degraded images, including background and body.

Contribution

The paper proposes leveraging facial degradation cues as an oracle to improve full-scene restoration in a diffusion-based framework, addressing limitations of existing methods.

Findings

Outperforms state-of-the-art scene restoration methods.

Effectively captures degradation attributes from facial cues.

Restores full scenes with high fidelity and consistency.

Abstract

Recent advances in image restoration have enabled high-fidelity recovery of faces from degraded inputs using reference-based face restoration models (Ref-FR). However, such methods focus solely on facial regions, neglecting degradation across the full scene, including body and background, which limits practical usability. Meanwhile, full-scene restorers often ignore degradation cues entirely, leading to underdetermined predictions and visual artifacts. In this work, we propose Face2Scene, a two-stage restoration framework that leverages the face as a perceptual oracle to estimate degradation and guide the restoration of the entire image. Given a degraded image and one or more identity references, we first apply a Ref-FR model to reconstruct high-quality facial details. From the restored-degraded face pair, we extract a face-derived degradation code that captures degradation attributes (e.g., noise, blur, compression), which is then transformed into multi-scale degradation-aware tokens. These tokens condition a diffusion model to restore the full scene in a single step, including the body and background. Extensive experiments demonstrate the superior effectiveness of the proposed method compared to state-of-the-art methods.