# AIM-SEEM: Adapting SEEM for Open-Vocabulary Terrain Segmentation Across Arbitrary Imaging Modalities

## TL;DR

This paper introduces AIM-SEEM, a framework that improves terrain segmentation for robots by adapting to various imaging modalities and expanding vocabulary.

## Contribution

The novel contribution is a unified framework for open-vocabulary terrain segmentation across arbitrary imaging modalities using a foundation model.

## Key findings

- AIM-SEEM outperforms prior methods in full-modality, modality-agnostic, and open-vocabulary settings.
- The framework introduces a vision-guided text calibration mechanism to handle distribution shifts in multi-modality inputs.
- Experiments demonstrate stable adaptation and controlled fusion of heterogeneous modalities.

## Abstract

Terrain segmentation performance directly affects the reliability of robotic environmental perception and decision making, yet most existing methods are built upon the assumptions of fixed sensing configurations and closed label sets. As a result, they struggle to meet real world outdoor requirements where modalities can be dynamically available and semantic classes continually expand. This paper systematically studies open-vocabulary terrain segmentation under arbitrary imaging modality combinations and proposes a unified foundation model-based framework named AIM-SEEM (SEEM for Arbitrary Imaging Modalities). Built upon Segment Everything Everywhere All at Once (SEEM), AIM-SEEM performs stable input side adaptation and controlled fusion of heterogeneous modalities, maximizing the reuse of pre-trained visual priors to accommodate different modality types and counts. Furthermore, to address the distribution shifts and the resulting vision–text alignment degradation caused by modality extension, a vision-guided text calibration mechanism is introduced to preserve open-vocabulary segmentation capability under multi-modality combination inputs. Experiments on two benchmarks under three evaluation settings, including full-modality, modality-agnostic, and open-vocabulary, show that AIM-SEEM consistently outperforms prior methods.