RemEdit: Efficient Diffusion Editing with Riemannian Geometry

TL;DR

RemEdit introduces a Riemannian geometry-based diffusion editing framework that achieves high-fidelity, real-time image edits by combining manifold navigation, dual-SLERP blending, and task-specific attention pruning.

Contribution

It presents a novel Riemannian manifold approach for diffusion-based image editing, improving fidelity and speed with efficient geodesic computation and token pruning.

Findings

Outperforms previous state-of-the-art editing methods.

Maintains real-time performance with over 50% token pruning.

Establishes new benchmarks for practical image editing.

Abstract

Controllable image generation is fundamental to the success of modern generative AI, yet it faces a critical trade-off between semantic fidelity and inference speed. The RemEdit diffusion-based framework addresses this trade-off with two synergistic innovations. First, for editing fidelity, we navigate the latent space as a Riemannian manifold. A mamba-based module efficiently learns the manifold's structure, enabling direct and accurate geodesic path computation for smooth semantic edits. This control is further refined by a dual-SLERP blending technique and a goal-aware prompt enrichment pass from a Vision-Language Model. Second, for additional acceleration, we introduce a novel task-specific attention pruning mechanism. A lightweight pruning head learns to retain tokens essential to the edit, enabling effective optimization without the semantic degradation common in content-agnostic approaches. RemEdit surpasses prior state-of-the-art editing frameworks while maintaining real-time performance under 50% pruning. Consequently, RemEdit establishes a new benchmark for practical and powerful image editing. Source code: https://www.github.com/eashanadhikarla/RemEdit.