Measuring the Intrinsic Dimension of Earth Representations
Arjun Rao, Marc Ru{\ss}wurm, Konstantin Klemmer, Esther Rolf
TL;DR
This paper investigates the intrinsic dimensionality of geographic implicit neural representations (INRs) for Earth data, revealing low-dimensional structures that correlate with task performance and can aid in model evaluation.
Contribution
It provides the first analysis of the intrinsic dimension of geographic INRs, offering a new, architecture-agnostic metric for assessing information content and model diagnostics.
Findings
Intrinsic dimensions of geographic INRs range between 2 and 10.
Intrinsic dimension correlates with downstream task performance.
Intrinsic dimension captures spatial artifacts and aids in model evaluation.
Abstract
Within the context of representation learning for Earth observation, geographic Implicit Neural Representations (INRs) embed low-dimensional location inputs (longitude, latitude) into high-dimensional embeddings, through models trained on geo-referenced satellite, image or text data. Despite the common aim of geographic INRs to distill Earth's data into compact, learning-friendly representations, we lack an understanding of how much information is contained in these Earth representations, and where that information is concentrated. The intrinsic dimension of a dataset measures the number of degrees of freedom required to capture its local variability, regardless of the ambient high-dimensional space in which it is embedded. This work provides the first study of the intrinsic dimensionality of geographic INRs. Analyzing INRs with ambient dimension between 256 and 512, we find that their…
Peer Reviews
Decision·ICLR 2026 Poster
1. The author proposed to use Intrinsic Dimension (ID) as a new metric to quantify the representativeness and task-lignment of the Earth representation. It gives us a way to interpret the location encoders and how the design can impact the model performance 2. A thorough analysis has been carried out to analysis the relation between ID and the task performance in the context of Earth presentation learning.
Although I enjoy reading this paper, there are some weaknesses I need to point out: 1. In the main results table (Table 1), the authors compared different pretrained location encoders with the global ID metric. However, different location encoders have different designs, different pretraining objectives, and different pretraining datasets and modalities. It is very hard to see any pattern here. A controlled experiment is needed in which the type of location encoders, the pretraining objectives,
1. It is important to introduce the concept of ID to the community of spatial representation learning. Unlike image or text embedding where there is **compression** of information, spatial representation such as location encoding usually embeds very low dimensional data (e.g. two-dimensional latitudes and longitudes) into high dimensional spaces, which is not a compression but an expansion. The quality of the expansion, i.e., whether the embedding manifold maintains useful topological informatio
1. The linear correlations fitted in Figure 3 and Figure 4 can be misleading. From the authors' perspective, they only need to show that the ID and model performance are positively/negatively correlated; the correlation does not need to be linear. For example, the subplots in Figure 3a look more quadratic/logarithmic than linear. 2. Table 1 can not be used to fairly compare the ID of baseline models. The models evaluated are trained on very different datasets and downstream tasks. For example,
[S1] The paper is the first to examine the intrinsic dimension of location embeddings, providing valuable insights into the representational limits and strengths of popular approaches. The study is especially useful given the large and growing number of works on location embeddings in GeoAI. [S2] The paper is well written and easy to follow. The methodology for estimating intrinsic dimension is clearly explained, and the results give interesting insights, including how intrinsic dimension varie
[W1] The TwoNN estimator is used to measure task alignment, with low ID values observed for task-specific representations. Can the authors elaborate on whether this could be an artifact of estimator bias due to non-uniform data, rather than actual compression? [W2] We observe notable differences in Global ID estimates across different estimators, especially for SatCLIP (e.g., SatCLIP-L40: 8.08 for FisherS vs. 2–2.5 for MLE, TLE, and TwoNN). The authors primarily focus on FisherS, which consiste
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSpatial Cognition and Navigation · Geographic Information Systems Studies · Automated Road and Building Extraction