HyNeuralMap: Hyperbolic Mapping of Visual Semantics to Neural Hierarchies

TL;DR

HyNeuralMap introduces a hyperbolic geometric framework to better align visual semantics with neural responses, capturing hierarchical structures more effectively than traditional Euclidean methods.

Contribution

The paper presents a novel hyperbolic Lorentz model for mapping visual semantics to neural hierarchies, improving semantic preservation and cross-subject neural similarity modeling.

Findings

Outperforms Euclidean baselines in semantic prediction tasks.

Effectively captures hierarchical semantic relationships.

Enhances cross-modal retrieval accuracy.

Abstract

Understanding the intricate mappings between visual stimuli and neural responses is a fundamental challenge in cognitive neuroscience. While current approaches predominantly align images and functional magnetic resonance imaging (fMRI) responses in Euclidean space, this geometry often struggles to preserve fine-grained semantic relationships and latent hierarchical structures across visual and neural modalities. To overcome this, we propose HyNeuralMap, a framework that employ hyperbolic Lorentz model to map visual semantics into a shared, cross-subject neural hierarchy. By leveraging the negative curvature of hyperbolic space as an inductive bias, the proposed framework better captures hierarchical semantic organization and cross-subject neural similarities. Specifically, visual and neural embeddings are jointly optimized through hyperbolic geometric alignment, where geodesic distances preserve semantic proximity and hierarchical relationships more effectively than Euclidean embeddings. Experiments demonstrate that HyNeuralMap consistently outperforms state-of-the-art Euclidean baselines in both multi-label semantic prediction and cross-modal retrieval tasks. This confirms hyperbolic geometry's superiority for cross-modal semantic alignment and hierarchical modeling, providing a new avenue for vision-neural representation learning.