CORA: Consistency-Guided Semi-Supervised Framework for Reasoning Segmentation

TL;DR

CORA is a semi-supervised framework that enhances reasoning segmentation by leveraging limited labeled data and unlabeled images through consistency-guided pseudo-label filtering and contrastive alignment, achieving state-of-the-art results with minimal supervision.

Contribution

The paper introduces CORA, a novel semi-supervised reasoning segmentation method that combines visual instructions, pseudo-label filtering, and contrastive alignment to improve performance under limited annotations.

Findings

Outperforms baselines with as few as 100 labeled images on Cityscapes.

Achieves +2.4% improvement with only 180 labeled images on PanNuke.

Demonstrates robustness under constrained annotation settings.

Abstract

Reasoning segmentation seeks pixel-accurate masks for targets referenced by complex, often implicit instructions, requiring context-dependent reasoning over the scene. Recent multimodal language models have advanced instruction following segmentation, yet generalization remains limited. The key bottleneck is the high cost of curating diverse, high-quality pixel annotations paired with rich linguistic supervision leading to brittle performance under distribution shift. Therefore, we present CORA, a semi-supervised reasoning segmentation framework that jointly learns from limited labeled data and a large corpus of unlabeled images. CORA introduces three main components: 1) conditional visual instructions that encode spatial and contextual relationships between objects; 2) a noisy pseudo-label filter based on the consistency of Multimodal LLM's outputs across semantically equivalent queries; and 3) a token-level contrastive alignment between labeled and pseudo-labeled samples to enhance feature consistency. These components enable CORA to perform robust reasoning segmentation with minimal supervision, outperforming existing baselines under constrained annotation settings. CORA achieves state-of-the-art results, requiring as few as 100 labeled images on Cityscapes, a benchmark dataset for urban scene understanding, surpassing the baseline by $+2.3\%$. Similarly, CORA improves performance by $+2.4\%$ with only 180 labeled images on PanNuke, a histopathology dataset.