Edge-Aware Image Manipulation via Diffusion Models with a Novel Structure-Preservation Loss

TL;DR

This paper introduces a Structure Preservation Loss (SPL) for diffusion models to better maintain edge structures during image editing, significantly improving structural fidelity in latent-diffusion-based tasks.

Contribution

The paper proposes a novel, training-free Structure Preservation Loss that enhances edge and structure preservation in diffusion-based image editing.

Findings

SPL improves structural fidelity in edited images.

The method achieves state-of-the-art results in latent-diffusion image editing.

The approach effectively preserves colors and local details.

Abstract

Recent advances in image editing leverage latent diffusion models (LDMs) for versatile, text-prompt-driven edits across diverse tasks. Yet, maintaining pixel-level edge structures-crucial for tasks such as photorealistic style transfer or image tone adjustment-remains as a challenge for latent-diffusion-based editing. To overcome this limitation, we propose a novel Structure Preservation Loss (SPL) that leverages local linear models to quantify structural differences between input and edited images. Our training-free approach integrates SPL directly into the diffusion model's generative process to ensure structural fidelity. This core mechanism is complemented by a post-processing step to mitigate LDM decoding distortions, a masking strategy for precise edit localization, and a color preservation loss to preserve hues in unedited areas. Experiments confirm SPL enhances structural fidelity, delivering state-of-the-art performance in latent-diffusion-based image editing. Our code will be publicly released at https://github.com/gongms00/SPL.