LORE: Latent Optimization for Precise Semantic Control in Rectified Flow-based Image Editing

TL;DR

LORE introduces a training-free latent optimization method for precise, controllable, and general-purpose image editing using rectified flow models, addressing semantic bias issues in existing inversion-based techniques.

Contribution

LORE proposes a novel, training-free approach that directly optimizes inverted noise for improved semantic control in image editing without architectural changes or fine-tuning.

Findings

LORE outperforms baselines in semantic alignment and image quality.

LORE demonstrates stability and scalability across benchmarks.

LORE effectively addresses semantic bias in rectified flow-based editing.

Abstract

Text-driven image editing enables users to flexibly modify visual content through natural language instructions, and is widely applied to tasks such as semantic object replacement, insertion, and removal. While recent inversion-based editing methods using rectified flow models have achieved promising results in image quality, we identify a structural limitation in their editing behavior: the semantic bias toward the source concept encoded in the inverted noise tends to suppress attention to the target concept. This issue becomes particularly critical when the source and target semantics are dissimilar, where the attention mechanism inherently leads to editing failure or unintended modifications in non-target regions. In this paper, we systematically analyze and validate this structural flaw, and introduce LORE, a training-free and efficient image editing method. LORE directly optimizes the inverted noise, addressing the core limitations in generalization and controllability of existing approaches, enabling stable, controllable, and general-purpose concept replacement, without requiring architectural modification or model fine-tuning. We conduct comprehensive evaluations on three challenging benchmarks: PIEBench, SmartEdit, and GapEdit. Experimental results show that LORE significantly outperforms strong baselines in terms of semantic alignment, image quality, and background fidelity, demonstrating the effectiveness and scalability of latent-space optimization for general-purpose image editing. Our implementation is available at https://github.com/oyly16/LORE.