EmoSpace: Fine-Grained Emotion Prototype Learning for Immersive Affective Content Generation

TL;DR

EmoSpace introduces a novel framework for fine-grained emotion-aware content generation in VR, utilizing dynamic emotion prototypes and hierarchical representations to enable nuanced emotional control without explicit labels.

Contribution

The paper proposes EmoSpace, a new method that learns interpretable emotion prototypes through vision-language alignment for immersive content creation.

Findings

Outperforms existing methods in qualitative and quantitative evaluations.

Enables diverse applications like emotional image outpainting and VR panorama generation.

User study shows VR environments influence emotional perception more than desktop settings.

Abstract

Emotion is important for creating compelling virtual reality (VR) content. Although some generative methods have been applied to lower the barrier to creating emotionally rich content, they fail to capture the nuanced emotional semantics and the fine-grained control essential for immersive experiences. To address these limitations, we introduce EmoSpace, a novel framework for emotion-aware content generation that learns dynamic, interpretable emotion prototypes through vision-language alignment. We employ a hierarchical emotion representation with rich learnable prototypes that evolve during training, enabling fine-grained emotional control without requiring explicit emotion labels. We develop a controllable generation pipeline featuring multi-prototype guidance, temporal blending, and attention reweighting that supports diverse applications, including emotional image outpainting, stylized generation, and emotional panorama generation for VR environments. Our experiments demonstrate the superior performance of EmoSpace over existing methods in both qualitative and quantitative evaluations. Additionally, we present a comprehensive user study investigating how VR environments affect emotional perception compared to desktop settings. Our work facilitates immersive visual content generation with fine-grained emotion control and supports applications like therapy, education, storytelling, artistic creation, and cultural preservation. Code and models will be made publicly available.