SPACE-CLIP: Spatial Perception via Adaptive CLIP Embeddings for Monocular Depth Estimation

TL;DR

SPACE-CLIP introduces a lightweight, decoder-only monocular depth estimation framework that leverages frozen CLIP vision features, enabling effective spatial perception without text prompts or backbone modifications.

Contribution

It presents a novel decoder-only approach that directly extracts geometric cues from a frozen CLIP vision encoder for monocular depth estimation.

Findings

Achieves state-of-the-art results on KITTI and NYU Depth V2 datasets.

Transfers effectively to a frozen SigLIP backbone with comparable performance.

Operates under text-free inference with a frozen vision backbone.

Abstract

Robotic and autonomous systems need dense spatial cues, but many monocular depth models are heavy, task-specific, or hard to attach to an existing multimodal stack. CLIP offers strong semantic representations, yet most CLIP-based depth methods still depend on text prompts or backbone updates, which complicate deployment in integrated control pipelines. We present SPACE-CLIP, a decoder-only depth framework that reads geometric cues directly from a frozen CLIP vision encoder and bypasses the text encoder at inference time. The model combines FiLM-conditioned semantic features from deep layers with structural features from shallow layers to recover both global scene layout and local geometric detail. Under the TFI-FB constraint (text-free inference and frozen vision backbone), SPACE-CLIP achieves AbsRel 0.0901 on KITTI and 0.1042 on NYU Depth V2, and the same dual-pathway decoder transfers to a frozen SigLIP backbone with comparable results. These findings show that a compact decoder can turn a shared foundation-model backbone into a reusable spatial perception module for embodied AI and autonomous robotic systems. Our model is available at https://github.com/taewan2002/space-clip