Intrinsic Lorentz Neural Network
Xianglong Shi, Ziheng Chen, Yunhan Jiang, Nicu Sebe
TL;DR
The paper introduces the Intrinsic Lorentz Neural Network (ILNN), a fully intrinsic hyperbolic neural network architecture that operates within the Lorentz model, improving performance on hierarchical data tasks.
Contribution
It proposes a novel intrinsic hyperbolic neural network architecture using Lorentz geometry, including a point-to-hyperplane layer and intrinsic modules, outperforming existing hyperbolic and Euclidean models.
Findings
ILNN achieves state-of-the-art results on CIFAR-10/100 and genomic benchmarks.
ILNN outperforms existing hyperbolic models in accuracy and efficiency.
Intrinsic modules like Lorentz batch normalization enhance training stability.
Abstract
Real-world data frequently exhibit latent hierarchical structures, which can be naturally represented by hyperbolic geometry. Although recent hyperbolic neural networks have demonstrated promising results, many existing architectures remain partially intrinsic, mixing Euclidean operations with hyperbolic ones or relying on extrinsic parameterizations. To address it, we propose the \emph{Intrinsic Lorentz Neural Network} (ILNN), a fully intrinsic hyperbolic architecture that conducts all computations within the Lorentz model. At its core, the network introduces a novel \emph{point-to-hyperplane} fully connected layer (FC), replacing traditional Euclidean affine logits with closed-form hyperbolic distances from features to learned Lorentz hyperplanes, thereby ensuring that the resulting geometric decision functions respect the inherent curvature. Around this fundamental layer, we design…
Peer Reviews
Decision·ICLR 2026 Poster
This paper's main contribution proposes a new point-to-hyperplane fully connected layer (PLFC) that utilizes intrinsic Lorentzian distance for logits to match the data's latent hierarchy. However, this approach is not originally developed by the authors and lacks novelty. Furthermore, the mechanism by which this method achieves matching with the data's latent hierarchy remains unclear and insufficiently explained.
(1) The paper claims to focus on real-world data with latent hierarchical structure and long-tail distribution. However, I did not find clear evidence that CIFAR-10, CIFAR-100, or the TEB and GUE genomics benchmarks possess such properties. (2) The PLFC method referenced by the authors as originating from HNN++ had been previously proposed in works on fully hyperbolic neural networks and the hypformer paper ("Hypformer: Exploring Efficient Transformer Fully in Hyperbolic Space"), yet the author
1. The proposed Intrinsic Lorentz Neural Network is well motivated as existing hyperbolic architectures remain partially intrinsic, mixing Euclidean operations with hyperbolic ones or relying on extrinsic parameterizations. 2. The proposed point-to-hyperplane Lorentz fully connected layer is new in the literature of hyperbolic neural network which replaces traditional affine transformations with intrinsic hyperbolic distance. 3. The paper is also generally well-written and extensive experiment
1. The experiments are only conducted on CIFAR-10/100 and two genomic benchmarks, it would be interesting to show if the proposed Intrinsic Lorentz Neural Network can outperform existing hyperbolic neural networks on large-scale datasets such as ImageNet and datasets with long-tail class distributions. 2. The authors argue that intrinsic designs are preferable compared with partially intrinsic or extrinsic parameterizations, but there is a lack of enough evidence showing that the claim is tru
1. The paper addresses an important topic of building neural architecture that respects hyperbolic geometry while avoiding the inconsistent mixture of Euclidean and hyperbolic components. 2. The experiments on vision and genomics tasks show potential interdisciplinary applications of hyperbolic neural networks. 3. On TEB and GUE datasets, ILNN achieves significant improvement on several tasks. 4. The paper is clearly structured, with detailed mathematical formulas and reasonable ablation studies
1. The paper claims that its method is “Intrinsic Lorentz”, yet provides no rigorous definition of what that means. If “intrinsic” means depending only on manifold operations, then computing directly in Minkowski coordinates is still extrinsic. 2. On CIFAR-10/100, the gain over Euclidean model is very limited (< 0.5%), and most of the selected baseline models are not even as good as the standard Euclidean model… 3. On TEB and GUE, there should be more baseline methods (some from what is reviewed
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsModel Reduction and Neural Networks · Advanced Graph Neural Networks · Tensor decomposition and applications