DynaGuide: A Generalizable Dynamic Guidance Framework for Unsupervised Semantic Segmentation

TL;DR

DynaGuide is a versatile unsupervised segmentation framework that combines global pseudo-labels with local boundary refinement, achieving state-of-the-art results without using ground-truth labels.

Contribution

It introduces a dual-guidance strategy with dynamic loss optimization, enabling high-precision segmentation by integrating diverse guidance sources in an unsupervised manner.

Findings

Achieves 17.5% mIoU improvement on BSD500

Improves mIoU by 3.1% on PASCAL VOC2012

Enhances mIoU by 11.66% on COCO

Abstract

Unsupervised image segmentation is a critical task in computer vision. It enables dense scene understanding without human annotations, which is especially valuable in domains where labelled data is scarce. However, existing methods often struggle to reconcile global semantic structure with fine-grained boundary accuracy. This paper introduces DynaGuide, an adaptive segmentation framework that addresses these challenges through a novel dual-guidance strategy and dynamic loss optimization. Building on our previous work, DynaSeg, DynaGuide combines global pseudo-labels from zero-shot models such as DiffSeg or SegFormer with local boundary refinement using a lightweight CNN trained from scratch. This synergy allows the model to correct coarse or noisy global predictions and produce high-precision segmentations. At the heart of DynaGuide is a multi-component loss that dynamically balances feature similarity, Huber-smoothed spatial continuity, including diagonal relationships, and semantic alignment with the global pseudo-labels. Unlike prior approaches, DynaGuide trains entirely without ground-truth labels in the target domain and supports plug-and-play integration of diverse guidance sources. Extensive experiments on BSD500, PASCAL VOC2012, and COCO demonstrate that DynaGuide achieves state-of-the-art performance, improving mIoU by 17.5% on BSD500, 3.1% on PASCAL VOC2012, and 11.66% on COCO. With its modular design, strong generalization, and minimal computational footprint, DynaGuide offers a scalable and practical solution for unsupervised segmentation in real-world settings. Code available at: https://github.com/RyersonMultimediaLab/DynaGuide