Towards Deeper Emotional Reflection: Crafting Affective Image Filters with Generative Priors

TL;DR

This paper introduces the Affective Image Filter task and models that transform text-based emotions into visually expressive images, leveraging generative priors to enhance emotional reflection and content fidelity.

Contribution

It presents new AIF models, including AIF-D with diffusion priors, for emotionally reflective image generation from text, advancing the state-of-the-art in affective image synthesis.

Findings

AIF models outperform existing methods in content and emotional accuracy.

AIF-D effectively leverages diffusion models for deeper emotional reflection.

User studies confirm AIF's superior ability to evoke targeted emotions.

Abstract

Social media platforms enable users to express emotions by posting text with accompanying images. In this paper, we propose the Affective Image Filter (AIF) task, which aims to reflect visually-abstract emotions from text into visually-concrete images, thereby creating emotionally compelling results. We first introduce the AIF dataset and the formulation of the AIF models. Then, we present AIF-B as an initial attempt based on a multi-modal transformer architecture. After that, we propose AIF-D as an extension of AIF-B towards deeper emotional reflection, effectively leveraging generative priors from pre-trained large-scale diffusion models. Quantitative and qualitative experiments demonstrate that AIF models achieve superior performance for both content consistency and emotional fidelity compared to state-of-the-art methods. Extensive user study experiments demonstrate that AIF models are significantly more effective at evoking specific emotions. Based on the presented results, we comprehensively discuss the value and potential of AIF models.