Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learning

TL;DR

This paper introduces pixel-level contrastive and propagation consistency tasks for unsupervised visual representation learning, significantly improving dense prediction performance over traditional instance-level methods.

Contribution

It proposes novel pixel-level pretext tasks that enhance dense feature learning and surpass state-of-the-art results in various downstream tasks.

Findings

Achieves 60.2 AP on Pascal VOC detection

Surpasses previous methods by 2.6 AP in object detection

Improves mIoU by 1.0 on Cityscapes segmentation

Abstract

Contrastive learning methods for unsupervised visual representation learning have reached remarkable levels of transfer performance. We argue that the power of contrastive learning has yet to be fully unleashed, as current methods are trained only on instance-level pretext tasks, leading to representations that may be sub-optimal for downstream tasks requiring dense pixel predictions. In this paper, we introduce pixel-level pretext tasks for learning dense feature representations. The first task directly applies contrastive learning at the pixel level. We additionally propose a pixel-to-propagation consistency task that produces better results, even surpassing the state-of-the-art approaches by a large margin. Specifically, it achieves 60.2 AP, 41.4 / 40.5 mAP and 77.2 mIoU when transferred to Pascal VOC object detection (C4), COCO object detection (FPN / C4) and Cityscapes semantic segmentation using a ResNet-50 backbone network, which are 2.6 AP, 0.8 / 1.0 mAP and 1.0 mIoU better than the previous best methods built on instance-level contrastive learning. Moreover, the pixel-level pretext tasks are found to be effective for pre-training not only regular backbone networks but also head networks used for dense downstream tasks, and are complementary to instance-level contrastive methods. These results demonstrate the strong potential of defining pretext tasks at the pixel level, and suggest a new path forward in unsupervised visual representation learning. Code is available at \url{https://github.com/zdaxie/PixPro}.