BotaCLIP: Contrastive Learning for Botany-Aware Representation of Earth Observation Data

TL;DR

BotaCLIP is a lightweight contrastive learning framework that adapts pre-trained Earth Observation models to incorporate botanical knowledge, improving ecological predictions without extensive retraining.

Contribution

It introduces a novel domain-specific adaptation method for foundation models that enhances ecological data analysis through contrastive learning with regularization.

Findings

Improved plant presence prediction accuracy

Enhanced butterfly occurrence modeling

Better soil trophic group estimation

Abstract

Foundation models have demonstrated a remarkable ability to learn rich, transferable representations across diverse modalities such as images, text, and audio. In modern machine learning pipelines, these representations often replace raw data as the primary input for downstream tasks. In this paper, we address the challenge of adapting a pre-trained foundation model to inject domain-specific knowledge, without retraining from scratch or incurring significant computational costs. To this end, we introduce BotaCLIP, a lightweight multimodal contrastive framework that adapts a pre-trained Earth Observation foundation model (DOFA) by aligning high-resolution aerial imagery with botanical relev\'es. Unlike generic embeddings, BotaCLIP internalizes ecological structure through contrastive learning with a regularization strategy that mitigates catastrophic forgetting. Once trained, the resulting embeddings serve as transferable representations for downstream predictors. Motivated by real-world applications in biodiversity modeling, we evaluated BotaCLIP representations in three ecological tasks: plant presence prediction, butterfly occurrence modeling, and soil trophic group abundance estimation. The results showed consistent improvements over those derived from DOFA and supervised baselines. More broadly, this work illustrates how domain-aware adaptation of foundation models can inject expert knowledge into data-scarce settings, enabling frugal representation learning.