EmotiCrafter: Text-to-Emotional-Image Generation based on Valence-Arousal Model

TL;DR

EmotiCrafter is a novel model that generates emotionally rich images from text prompts by embedding continuous Valence-Arousal emotion values, enabling nuanced emotion control and content accuracy.

Contribution

The paper introduces C-EICG, a new task for continuous emotional image generation, and proposes EmotiCrafter with an emotion-embedding network and specialized loss function.

Findings

Effectively generates images with specific emotions and content.

Outperforms existing emotional image generation methods.

Successfully captures complex emotional nuances.

Abstract

Recent research shows that emotions can enhance users' cognition and influence information communication. While research on visual emotion analysis is extensive, limited work has been done on helping users generate emotionally rich image content. Existing work on emotional image generation relies on discrete emotion categories, making it challenging to capture complex and subtle emotional nuances accurately. Additionally, these methods struggle to control the specific content of generated images based on text prompts. In this work, we introduce the new task of continuous emotional image content generation (C-EICG) and present EmotiCrafter, an emotional image generation model that generates images based on text prompts and Valence-Arousal values. Specifically, we propose a novel emotion-embedding mapping network that embeds Valence-Arousal values into textual features, enabling the capture of specific emotions in alignment with intended input prompts. Additionally, we introduce a loss function to enhance emotion expression. The experimental results show that our method effectively generates images representing specific emotions with the desired content and outperforms existing techniques.