Annotation Free Spacecraft Detection and Segmentation using Vision Language Models

TL;DR

This paper introduces an annotation-free method for detecting and segmenting spacecraft in space images by leveraging vision language models and pseudo-labeling, significantly reducing manual annotation efforts.

Contribution

The authors develop a novel pipeline that uses VLMs to generate pseudo-labels and train lightweight models via label distillation, enabling effective space target segmentation without manual annotations.

Findings

Achieved up to 10-point improvement in average precision on multiple datasets.

Demonstrated the effectiveness of pseudo-labeling with noisy labels in space imagery.

Validated the approach across diverse space datasets.

Abstract

Vision Language Models (VLMs) have demonstrated remarkable performance in open-world zero-shot visual recognition. However, their potential in space-related applications remains largely unexplored. In the space domain, accurate manual annotation is particularly challenging due to factors such as low visibility, illumination variations, and object blending with planetary backgrounds. Developing methods that can detect and segment spacecraft and orbital targets without requiring extensive manual labeling is therefore of critical importance. In this work, we propose an annotation-free detection and segmentation pipeline for space targets using VLMs. Our approach begins by automatically generating pseudo-labels for a small subset of unlabeled real data with a pre-trained VLM. These pseudo-labels are then leveraged in a teacher-student label distillation framework to train lightweight models. Despite the inherent noise in the pseudo-labels, the distillation process leads to substantial performance gains over direct zero-shot VLM inference. Experimental evaluations on the SPARK-2024, SPEED+, and TANGO datasets on segmentation tasks demonstrate consistent improvements in average precision (AP) by up to 10 points. Code and models are available at https://github.com/giddyyupp/annotation-free-spacecraft-segmentation.