Measurement-Constrained Sampling for Text-Prompted Blind Face Restoration

TL;DR

This paper introduces a measurement-constrained sampling method for blind face restoration that produces diverse, prompt-aligned high-quality face images from extremely low-quality inputs, addressing the one-to-many nature of the task.

Contribution

It formulates blind face restoration as a measurement-constrained generative problem, enabling diverse reconstructions conditioned on textual prompts using diffusion models.

Findings

Outperforms existing BFR methods in prompt alignment and quality.

Generates diverse high-quality face images from low-quality inputs.

Effectively captures the one-to-many nature of face restoration.

Abstract

Blind face restoration (BFR) may correspond to multiple plausible high-quality (HQ) reconstructions under extremely low-quality (LQ) inputs. However, existing methods typically produce deterministic results, struggling to capture this one-to-many nature. In this paper, we propose a Measurement-Constrained Sampling (MCS) approach that enables diverse LQ face reconstructions conditioned on different textual prompts. Specifically, we formulate BFR as a measurement-constrained generative task by constructing an inverse problem through controlled degradations of coarse restorations, which allows posterior-guided sampling within text-to-image diffusion. Measurement constraints include both Forward Measurement, which ensures results align with input structures, and Reverse Measurement, which produces projection spaces, ensuring that the solution can align with various prompts. Experiments show that our MCS can generate prompt-aligned results and outperforms existing BFR methods. Codes will be released after acceptance.