HEART: Hyperspherical Embedding Alignment via Kent-Representation Traversal in Diffusion Models

TL;DR

HEART introduces a geometry-aware approach using hyperspherical embeddings and Kent distributions to improve control over diffusion model outputs without fine-tuning.

Contribution

This work reveals the hyperspherical structure of text embeddings and proposes a Kent-distribution-based method for more precise, geometry-respecting image editing in diffusion models.

Findings

Enables consistent subject replacement and attribute control

Requires no finetuning, inversion, or optimization

Generalizes across different diffusion architectures

Abstract

Text-to-image diffusion models can generate visually stunning images, yet, controlling what appears and how it appears, remains surprisingly difficult, especially when operating solely within the constraints of the text-conditioning space. For example, changing a subject or adjusting an attribute often leads to unintended side effects, such as altered backgrounds or distorted details. This is because most existing text-based control methods treat the embedding space as Euclidean and apply simple linear transformations, which do not reflect how semantic concepts are actually organized. In this work, we take a step back and ask: what is the true geometry of these embeddings? We find that text encoder representations lie on a hypersphere, where concepts are not linear directions but structured, anisotropic distributions better captured by Kent distributions. Building on this insight, we propose HEART, a training-free framework that performs Kent-aware geodesic transformations directly on the hypersphere. By respecting the underlying geometry, HEART enables intuitive and precise edits, such as consistent subject replacement and fine-grained attribute control, while preserving the original scene. Importantly, HEART requires no finetuning, inversion, or optimization, and generalizes across diffusion model architectures. Our results show that a simple shift in perspective, from linear to spherical, can unlock fast, and controllable image generation.