Make Me Happier: Evoking Emotions Through Image Diffusion Models

TL;DR

This paper introduces a diffusion-based method for emotional image editing that preserves image semantics while evoking target emotions, supported by a new large-scale dataset and comprehensive benchmarking.

Contribution

We propose a novel diffusion model for emotion-evoked image generation and provide a large emotion-annotated dataset, addressing the lack of existing resources and benchmarks.

Findings

Our method outperforms baseline approaches in evoking target emotions.

The diffusion model effectively identifies emotional cues and preserves image structure.

Experimental results validate the approach's superiority in emotional image editing.

Abstract

Despite the rapid progress in image generation, emotional image editing remains under-explored. The semantics, context, and structure of an image can evoke emotional responses, making emotional image editing techniques valuable for various real-world applications, including treatment of psychological disorders, commercialization of products, and artistic design. First, we present a novel challenge of emotion-evoked image generation, aiming to synthesize images that evoke target emotions while retaining the semantics and structures of the original scenes. To address this challenge, we propose a diffusion model capable of effectively understanding and editing source images to convey desired emotions and sentiments. Moreover, due to the lack of emotion editing datasets, we provide a unique dataset consisting of 340,000 pairs of images and their emotion annotations. Furthermore, we conduct human psychophysics experiments and introduce a new evaluation metric to systematically benchmark all the methods. Experimental results demonstrate that our method surpasses all competitive baselines. Our diffusion model is capable of identifying emotional cues from original images, editing images that elicit desired emotions, and meanwhile, preserving the semantic structure of the original images. All code, model, and dataset are available at GitHub.