CogniEdit: Dense Gradient Flow Optimization for Fine-Grained Image Editing

TL;DR

CogniEdit introduces a unified dense gradient flow optimization framework for fine-grained image editing, leveraging multi-modal reasoning and trajectory-level supervision to improve precision and quality in diffusion-based edits.

Contribution

It combines multi-modal large language models, dynamic token focus, and dense gradient optimization to enhance fine-grained image editing capabilities beyond existing sparse feedback methods.

Findings

Achieves state-of-the-art results on benchmark datasets.

Balances fine-grained instruction following with visual quality.

Enables trajectory-level control through dense gradient propagation.

Abstract

Instruction-based image editing with diffusion models has achieved impressive results, yet existing methods struggle with fine-grained instructions specifying precise attributes such as colors, positions, and quantities. While recent approaches employ Group Relative Policy Optimization (GRPO) for alignment, they optimize only at individual sampling steps, providing sparse feedback that limits trajectory-level control. We propose a unified framework CogniEdit, combining multi-modal reasoning with dense reward optimization that propagates gradients across consecutive denoising steps, enabling trajectory-level gradient flow through the sampling process. Our method comprises three components: (1) Multi-modal Large Language Models for decomposing complex instructions into actionable directives, (2) Dynamic Token Focus Relocation that adaptively emphasizes fine-grained attributes, and (3) Dense GRPO-based optimization that propagates gradients across consecutive steps for trajectory-level supervision. Extensive experiments on benchmark datasets demonstrate that our CogniEdit achieves state-of-the-art performance in balancing fine-grained instruction following with visual quality and editability preservation