MILD: Multi-Layer Diffusion Strategy for Complex and Precise Multi-IP Aware Human Erasing

TL;DR

This paper introduces MILD, a multi-layer diffusion approach with new datasets and modules for precise, multi-instance human erasing in complex images, significantly improving over existing methods.

Contribution

The paper proposes a novel multi-layer diffusion strategy with spatial modules and a new dataset to enhance multi-instance human erasing in complex scenarios.

Findings

Outperforms existing human erasing methods.

Effectively handles occlusion and entanglement.

Reduces boundary artifacts and semantic leakage.

Abstract

Recent years have witnessed the success of diffusion models in image customization tasks. However, existing mask-guided human erasing methods still struggle in complex scenarios such as human-human occlusion, human-object entanglement, and human-background interference, mainly due to the lack of large-scale multi-instance datasets and effective spatial decoupling to separate foreground from background. To bridge these gaps, we curate the MILD dataset capturing diverse poses, occlusions, and complex multi-instance interactions. We then define the Cross-Domain Attention Gap (CAG), an attention-gap metric to quantify semantic leakage. On top of these, we propose Multi-Layer Diffusion (MILD), which decomposes the generation process into independent denoising pathways, enabling separate reconstruction of each foreground instance and the background. To enhance human-centric understanding, we introduce Human Morphology Guidance, a plug-and-play module that incorporates pose, parsing, and spatial relationships into the diffusion process to improve structural awareness and restoration quality. Additionally, we present Spatially-Modulated Attention, an adaptive mechanism that leverages spatial mask priors to modulate attention across semantic regions, further widening the CAG to effectively minimize boundary artifacts and mitigate semantic leakage. Experiments show that MILD significantly outperforms existing methods. Datasets and code are publicly available at: https://mild-multi-layer-diffusion.github.io/.