EmoLat: Text-driven Image Sentiment Transfer via Emotion Latent Space

TL;DR

EmoLat introduces a cross-modal emotion latent space for fine-grained, text-driven image sentiment transfer, leveraging an emotion semantic graph and adversarial regularization to improve transfer quality and controllability.

Contribution

The paper presents EmoLat, a novel emotion latent space with an emotion semantic graph and adversarial regularization, enabling improved text-driven image sentiment transfer.

Findings

Outperforms existing methods in transfer fidelity

Introduces EmoSpace Set, a large-scale emotion-annotated image dataset

Demonstrates effective cross-modal sentiment manipulation

Abstract

We propose EmoLat, a novel emotion latent space that enables fine-grained, text-driven image sentiment transfer by modeling cross-modal correlations between textual semantics and visual emotion features. Within EmoLat, an emotion semantic graph is constructed to capture the relational structure among emotions, objects, and visual attributes. To enhance the discriminability and transferability of emotion representations, we employ adversarial regularization, aligning the latent emotion distributions across modalities. Building upon EmoLat, a cross-modal sentiment transfer framework is proposed to manipulate image sentiment via joint embedding of text and EmoLat features. The network is optimized using a multi-objective loss incorporating semantic consistency, emotion alignment, and adversarial regularization. To support effective modeling, we construct EmoSpace Set, a large-scale benchmark dataset comprising images with dense annotations on emotions, object semantics, and visual attributes. Extensive experiments on EmoSpace Set demonstrate that our approach significantly outperforms existing state-of-the-art methods in both quantitative metrics and qualitative transfer fidelity, establishing a new paradigm for controllable image sentiment editing guided by textual input. The EmoSpace Set and all the code are available at http://github.com/JingVIPLab/EmoLat.