LCUDiff: Latent Capacity Upgrade Diffusion for Faithful Human Body Restoration

TL;DR

LCUDiff introduces a novel one-step latent diffusion framework that upgrades pre-trained models from 4-channel to 16-channel latent spaces, significantly improving human body restoration fidelity while maintaining efficiency.

Contribution

The paper presents a new method for latent space upgrade in diffusion models, including channel splitting distillation, prior-preserving adaptation, and a decoder router for enhanced human body image restoration.

Findings

Achieves higher fidelity in human body restoration with fewer artifacts.

Maintains one-step efficiency in the diffusion process.

Demonstrates superior performance on synthetic and real-world datasets.

Abstract

Existing methods for restoring degraded human-centric images often struggle with insufficient fidelity, particularly in human body restoration (HBR). Recent diffusion-based restoration methods commonly adapt pre-trained text-to-image diffusion models, where the variational autoencoder (VAE) can significantly bottleneck restoration fidelity. We propose LCUDiff, a stable one-step framework that upgrades a pre-trained latent diffusion model from the 4-channel latent space to the 16-channel latent space. For VAE fine-tuning, channel splitting distillation (CSD) is used to keep the first four channels aligned with pre-trained priors while allocating the additional channels to effectively encode high-frequency details. We further design prior-preserving adaptation (PPA) to smoothly bridge the mismatch between 4-channel diffusion backbones and the higher-dimensional 16-channel latent. In addition, we propose a decoder router (DeR) for per-sample decoder routing using restoration-quality score annotations, which improves visual quality across diverse conditions. Experiments on synthetic and real-world datasets show competitive results with higher fidelity and fewer artifacts under mild degradations, while preserving one-step efficiency. The code and model will be at https://github.com/gobunu/LCUDiff.