Towards Non-Euclidean Foundation Models: Advancing AI Beyond Euclidean Frameworks

TL;DR

This paper discusses the development of foundation models that utilize non-Euclidean geometries like hyperbolic and spherical spaces to better capture complex data structures, especially in web-related applications.

Contribution

It introduces the concept of Non-Euclidean Foundation Models and explores their potential to overcome limitations of Euclidean-based models in representing complex data.

Findings

Non-Euclidean spaces improve data representation efficiency.

Integration with foundation models enhances modeling of web data.

Potential for improved performance in search and recommendation systems.

Abstract

In the era of foundation models and Large Language Models (LLMs), Euclidean space is the de facto geometric setting of our machine learning architectures. However, recent literature has demonstrated that this choice comes with fundamental limitations. To that end, non-Euclidean learning is quickly gaining traction, particularly in web-related applications where complex relationships and structures are prevalent. Non-Euclidean spaces, such as hyperbolic, spherical, and mixed-curvature spaces, have been shown to provide more efficient and effective representations for data with intrinsic geometric properties, including web-related data like social network topology, query-document relationships, and user-item interactions. Integrating foundation models with non-Euclidean geometries has great potential to enhance their ability to capture and model the underlying structures, leading to better performance in search, recommendations, and content understanding. This workshop focuses on the intersection of Non-Euclidean Foundation Models and Geometric Learning (NEGEL), exploring its potential benefits, including the potential benefits for advancing web-related technologies, challenges, and future directions. Workshop page: [https://hyperboliclearning.github.io/events/www2025workshop](https://hyperboliclearning.github.io/events/www2025workshop)