Unsupervised Discovery of the Long-Tail in Instance Segmentation Using Hierarchical Self-Supervision

TL;DR

This paper introduces an unsupervised method for discovering long-tail object categories in instance segmentation by learning hierarchical self-supervised embeddings, enabling the identification of novel and fine-grained objects without additional annotations.

Contribution

It proposes a novel hierarchical self-supervised learning approach for unsupervised long-tail category discovery in instance segmentation, reducing reliance on annotated datasets.

Findings

Achieves competitive results on LVIS dataset without extra annotations.

Discovers more fine-grained and novel objects than standard categories.

Effectively leverages hierarchical relationships for self-supervised learning.

Abstract

Instance segmentation is an active topic in computer vision that is usually solved by using supervised learning approaches over very large datasets composed of object level masks. Obtaining such a dataset for any new domain can be very expensive and time-consuming. In addition, models trained on certain annotated categories do not generalize well to unseen objects. The goal of this paper is to propose a method that can perform unsupervised discovery of long-tail categories in instance segmentation, through learning instance embeddings of masked regions. Leveraging rich relationship and hierarchical structure between objects in the images, we propose self-supervised losses for learning mask embeddings. Trained on COCO dataset without additional annotations of the long-tail objects, our model is able to discover novel and more fine-grained objects than the common categories in COCO. We show that the model achieves competitive quantitative results on LVIS as compared to the supervised and partially supervised methods.