Split-Fuse-Transport: Annotation-Free Saliency via Dual Clustering and Optimal Transport Alignment

TL;DR

AutoSOD introduces an unsupervised salient object detection method that leverages dual clustering and optimal transport to generate high-quality pseudo-masks, significantly improving accuracy without pixel-level labels.

Contribution

The paper proposes POTNet with dual clustering and optimal transport, enabling near-supervised SOD accuracy in an end-to-end unsupervised pipeline without handcrafted priors.

Findings

Outperforms existing unsupervised methods by up to 26% in F-measure.

Outperforms weakly supervised methods by up to 36% in F-measure.

Narrowing the gap to fully supervised models.

Abstract

Salient object detection (SOD) aims to segment visually prominent regions in images and serves as a foundational task for various computer vision applications. We posit that SOD can now reach near-supervised accuracy without a single pixel-level label, but only when reliable pseudo-masks are available. We revisit the prototype-based line of work and make two key observations. First, boundary pixels and interior pixels obey markedly different geometry; second, the global consistency enforced by optimal transport (OT) is underutilized if prototype quality is weak. To address this, we introduce POTNet, an adaptation of Prototypical Optimal Transport that replaces POT's single k-means step with an entropy-guided dual-clustering head: high-entropy pixels are organized by spectral clustering, low-entropy pixels by k-means, and the two prototype sets are subsequently aligned by OT. This split-fuse-transport design yields sharper, part-aware pseudo-masks in a single forward pass, without handcrafted priors. Those masks supervise a standard MaskFormer-style encoder-decoder, giving rise to AutoSOD, an end-to-end unsupervised SOD pipeline that eliminates SelfMask's offline voting yet improves both accuracy and training efficiency. Extensive experiments on five benchmarks show that AutoSOD outperforms unsupervised methods by up to 26% and weakly supervised methods by up to 36% in F-measure, further narrowing the gap to fully supervised models.