What on Earth is AlphaEarth? Hierarchical structure and functional interpretability for global land cover

Technical Deep Dive: What on Earth is AlphaEarth?

Overview

This paper proposes a functional interpretability framework for understanding the internal structure and organization of geospatial foundation model (GFM) embeddings, using the Google AlphaEarth Foundations (GAEF) model as a case study. The key findings are:

• GAEF embeddings exhibit a non-uniform functional structure, with dimensions playing differentiated roles ranging from high specialization in particular land cover classes to the encoding of broader, shared environmental gradients.
• Embedding dimensions can be categorized into specialist, low-generalist, mid-generalist, and high-generalist types based on their contributions to land cover discrimination.
• Specialist embeddings capture the distinctive characteristics of "core" land cover classes, while generalist embeddings represent relationships and transitions between classes.
• The hierarchical functional organization of the embedding space provides a pathway toward systematic, task-independent interpretation of what the embeddings encode, going beyond associations with individual physical variables.

Problem & Context

Geospatial foundation models like GAEF integrate multiple Earth observation data sources into high-dimensional embeddings that achieve strong predictive performance across a variety of mapping and environmental modeling tasks. However, the internal organization of these embedding spaces remains opaque, limiting their scientific use and integration into high-stakes decision-making.

Unlike conventional remote sensing products, GAEF embeddings do not have a direct correspondence to measurable physical properties. Their abstract, high-dimensional representation poses a significant interpretability challenge, as the environmental information they encode is not readily trackable or reproducible.

To address this gap, the authors propose a functional interpretability framework that characterizes the role of embedding dimensions in representing land cover organization, revealing a structured hierarchy from specialist to generalist dimensions.

Methodology

The study combines two key components:

1. Massive Experimental Exploration:
   • Executed over 130,000 binary classification experiments targeting each of the 11 ESA WorldCover 2020 land cover classes.
   • Used four machine learning algorithms (Random Forest, Gradient Boosting, XGBoost, LightGBM) with progressive feature ablation.
   • Identified the minimum number of embedding dimensions required to achieve 98% of baseline classification performance for each class.
2. Structural Analysis of the Embedding Space:
   • Constructed an association matrix quantifying the relevance of each embedding dimension for each land cover class.
   • Classified dimensions as specialist, low-generalist, mid-generalist, or high-generalist based on the number of classes they contribute to.
   • Developed an "Embedding Fingerprint" visualization to summarize the dimensional composition of each land cover class.
   • Created an "Embedding Universe" visualization to illustrate the functional organization of the latent space.

Data & Experimental Setup

The analysis was based on two main datasets:

1. Land Cover Labels: ESA WorldCover 2020 product, providing a global 10-meter resolution classification into 11 discrete land cover categories.
2. Geospatial Embeddings: 64-dimensional Google AlphaEarth Foundations (GAEF) embeddings, integrating multi-modal Earth observation data.

The experiments were designed to systematically evaluate the discriminatory capacity of the embeddings in land cover classification tasks, focusing on the minimum number of dimensions required to achieve 98% of baseline performance.

Results

The key findings of the study include:

1. Informational Efficiency of Embeddings:
   • Accurate land cover classification (≥98% of baseline) can be achieved using as few as 2 to 12 embedding dimensions, depending on the class.
   • This demonstrates substantial redundancy in the embedding space and offers a pathway toward significant reductions in computational cost.
2. Emergence of Specialized Embedding Dimensions:
   • Certain embedding dimensions exhibit a high concentration of importance for specific land cover classes, capturing their distinctive physical signatures.
   • These specialist embeddings represent the "core" of land cover units, enabling accurate discrimination.
3. Identification of Shared Embedding Dimensions:
   • Some embedding dimensions make significant contributions across multiple land cover classes, suggesting they capture common patterns or interactions between classes.
   • These shared, generalist embeddings likely represent transition zones or ecotones where land cover characteristics overlap.
4. Hierarchical Functional Organization of the Embedding Space:
   • The embedding space exhibits a structured hierarchy, with dimensions playing differentiated roles ranging from high specialization to broad generalization.
   • Specialist embeddings, low- and mid-generalist embeddings, and high-generalist embeddings collectively form a functional spectrum that reflects the organization of the Earth's surface.

Interpretation

The findings suggest that the GAEF embedding space is not an arbitrary representation, but rather encodes a structured blueprint of geographic organization:

• Specialist Embeddings: Capture the distinctive characteristics of "core" land cover units, enabling accurate discrimination.
• Low- and Mid-Generalist Embeddings: Represent relationships and transitions between land cover classes, reflecting ecotonal boundaries and spatial interactions.
• High-Generalist Embeddings: Encode broader environmental gradients and patterns that transcend individual land cover categories.

This hierarchical functional organization provides a pathway toward systematic, task-independent interpretation of what the embeddings encode, going beyond associations with individual physical variables.

Limitations & Uncertainties

• The interpretations of embedding dimensions depend, to some extent, on the machine learning algorithms and data distribution used in the experimental setup.
• The functional classification of dimensions is derived from a 98% baseline performance recovery threshold, and the sensitivity of results to this choice warrants further analysis.
• The 2D visualizations represent a simplification of the high-dimensional embedding space, and regional variations in embedding discriminatory capacity were observed.
• The study does not directly associate embedding dimensions with specific physical variables or environmental processes, which constitutes an important direction for future research.

What Comes Next

The authors propose several future research directions:

1. Validating the proposed interpretations by associating embeddings with specific physical variables and environmental processes.
2. Analyzing the stability of the functional classification across different regions and temporal scales.
3. Investigating the geographic determinants of embedding discriminatory capacity and how they vary across biomes or climatic zones.
4. Extending the framework to other remote sensing tasks, such as change detection, semantic segmentation, or environmental variable prediction.
5. Integrating these results into interactive analysis and decision-making systems to bring foundation models closer to practical applications.

Overall, this work lays the groundwork for a more systematic and interpretable use of geospatial foundation models in scientific and applied contexts, bridging the gap between their predictive performance and their understanding.