SEMPART: Self-supervised Multi-resolution Partitioning of Image Semantics

TL;DR

SEMPART is a self-supervised method that efficiently produces high-quality, multi-resolution semantic masks for salient object detection by jointly inferring coarse and fine partitions with boundary preservation.

Contribution

It introduces a novel joint inference framework for coarse and fine image partitions using DINO-based semantics, enhancing object segmentation without extra post-processing.

Findings

Produces high-quality masks rapidly

Benefits from co-optimizing coarse and fine branches

Outperforms existing methods in salient object detection

Abstract

Accurately determining salient regions of an image is challenging when labeled data is scarce. DINO-based self-supervised approaches have recently leveraged meaningful image semantics captured by patch-wise features for locating foreground objects. Recent methods have also incorporated intuitive priors and demonstrated value in unsupervised methods for object partitioning. In this paper, we propose SEMPART, which jointly infers coarse and fine bi-partitions over an image's DINO-based semantic graph. Furthermore, SEMPART preserves fine boundary details using graph-driven regularization and successfully distills the coarse mask semantics into the fine mask. Our salient object detection and single object localization findings suggest that SEMPART produces high-quality masks rapidly without additional post-processing and benefits from co-optimizing the coarse and fine branches.