Affective Image Editing: Shaping Emotional Factors via Text Descriptions

TL;DR

AIEdiT is a novel framework for affective image editing that uses text descriptions to evoke specific emotions by adaptively shaping emotional factors across images, supported by a large-scale emotion-aligned dataset.

Contribution

The paper introduces a new affective image editing method using text, including a continuous emotional spectrum, emotional mapper, and supervised training with MLLM, along with a large-scale dataset.

Findings

AIEdiT effectively evokes targeted emotions in images.

The method outperforms existing approaches in emotional accuracy.

Extensive experiments validate the approach's effectiveness.

Abstract

In daily life, images as common affective stimuli have widespread applications. Despite significant progress in text-driven image editing, there is limited work focusing on understanding users' emotional requests. In this paper, we introduce AIEdiT for Affective Image Editing using Text descriptions, which evokes specific emotions by adaptively shaping multiple emotional factors across the entire images. To represent universal emotional priors, we build the continuous emotional spectrum and extract nuanced emotional requests. To manipulate emotional factors, we design the emotional mapper to translate visually-abstract emotional requests to visually-concrete semantic representations. To ensure that editing results evoke specific emotions, we introduce an MLLM to supervise the model training. During inference, we strategically distort visual elements and subsequently shape corresponding emotional factors to edit images according to users' instructions. Additionally, we introduce a large-scale dataset that includes the emotion-aligned text and image pair set for training and evaluation. Extensive experiments demonstrate that AIEdiT achieves superior performance, effectively reflecting users' emotional requests.