Colorization as a Proxy Task for Visual Understanding
Gustav Larsson, Michael Maire, Gregory Shakhnarovich

TL;DR
This paper explores using automatic colorization as a self-supervised proxy task to improve visual understanding, achieving state-of-the-art results without ImageNet labels and analyzing factors influencing its effectiveness.
Contribution
It demonstrates that colorization as a self-supervised task can match ImageNet pretraining in various tasks and provides an in-depth analysis of factors affecting its success.
Findings
State-of-the-art results on VOC segmentation and classification without ImageNet labels
Analysis of loss formulation, training details, and architecture impacts
Colorization offers a supervisory signal comparable to ImageNet pretraining
Abstract
We investigate and improve self-supervision as a drop-in replacement for ImageNet pretraining, focusing on automatic colorization as the proxy task. Self-supervised training has been shown to be more promising for utilizing unlabeled data than other, traditional unsupervised learning methods. We build on this success and evaluate the ability of our self-supervised network in several contexts. On VOC segmentation and classification tasks, we present results that are state-of-the-art among methods not using ImageNet labels for pretraining representations. Moreover, we present the first in-depth analysis of self-supervision via colorization, concluding that formulation of the loss, training details and network architecture play important roles in its effectiveness. This investigation is further expanded by revisiting the ImageNet pretraining paradigm, asking questions such as: How much…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Chemical Sensor Technologies · Advanced Image and Video Retrieval Techniques · Domain Adaptation and Few-Shot Learning
MethodsColorization
