IC-EO: Interpretable Code-based assistant for Earth Observation

TL;DR

This paper introduces IC-EO, an interpretable, code-based conversational agent that transforms natural language queries into executable Python workflows for Earth Observation analysis, enhancing transparency and reproducibility.

Contribution

It presents a novel framework combining tool LLMs with a unified API to generate auditable code for EO tasks, outperforming baseline models in accuracy and interpretability.

Findings

Achieves 64.2% accuracy on land-composition mapping

Outperforms baselines like GPT-4o and LLaVA in EO tasks

Produces transparent, verifiable code for EO analysis

Abstract

Despite recent advances in computer vision, Earth Observation (EO) analysis remains difficult to perform for the laymen, requiring expert knowledge and technical capabilities. Furthermore, many systems return black-box predictions that are difficult to audit or reproduce. Leveraging recent advances in tool LLMs, this study proposes a conversational, code-generating agent that transforms natural-language queries into executable, auditable Python workflows. The agent operates over a unified easily extendable API for classification, segmentation, detection (oriented bounding boxes), spectral indices, and geospatial operators. With our proposed framework, it is possible to control the results at three levels: (i) tool-level performance on public EO benchmarks; (ii) at the agent-level to understand the capacity to generate valid, hallucination-free code; and (iii) at the task-level on specific use cases. In this work, we select two use-cases of interest: land-composition mapping and post-wildfire damage assessment. The proposed agent outperforms general-purpose LLM/VLM baselines (GPT-4o, LLaVA), achieving 64.2% vs. 51.7% accuracy on land-composition and 50% vs. 0% on post-wildfire analysis, while producing results that are transparent and easy to interpret. By outputting verifiable code, the approach turns EO analysis into a transparent, reproducible process.