Multiple Object Stitching for Unsupervised Representation Learning

TL;DR

This paper introduces Multiple Object Stitching (MOS), a simple unsupervised method that enhances multi-object image representations by stitching single-object images, improving performance on complex downstream tasks without requiring human annotations.

Contribution

MOS is a novel stitching-based approach that refines unsupervised representations for multi-object images, leveraging object correspondences without human labels.

Findings

Achieves state-of-the-art unsupervised performance on ImageNet, CIFAR, and COCO.

Improves object detection and segmentation tasks.

Effective on both single-object and multi-object images.

Abstract

Contrastive learning for single object centric images has achieved remarkable progress on unsupervised representation, but suffering inferior performance on the widespread images with multiple objects. In this paper, we propose a simple but effective method, Multiple Object Stitching (MOS), to refine the unsupervised representation for multi-object images. Specifically, we construct the multi-object images by stitching the single object centric ones, where the objects in the synthesized multi-object images are predetermined. Hence, compared to the existing contrastive methods, our method provides additional object correspondences between multi-object images without human annotations. In this manner, our method pays more attention to the representations of each object in multi-object image, thus providing more detailed representations for complicated downstream tasks, such as object detection and semantic segmentation. Experimental results on ImageNet, CIFAR and COCO datasets demonstrate that our proposed method achieves the leading unsupervised representation performance on both single object centric images and multi-object ones. The source code is available at https://github.com/visresearch/MultipleObjectStitching.