Improving task-specific representation via 1M unlabelled images without any extra knowledge

TL;DR

This paper demonstrates that conditioning existing models on a large, diverse set of unlabelled images enhances task-specific representations, improving surface normal estimation and semantic segmentation without extra knowledge or architecture changes.

Contribution

It introduces a simple method of leveraging unlabelled images to improve task-specific models without additional knowledge or modifications.

Findings

Improved surface normal estimation by 4% on NYU-v2.

Enhanced semantic segmentation by 4% on PASCAL VOC.

Method requires no extra knowledge or architecture changes.

Abstract

We present a case-study to improve the task-specific representation by leveraging a million unlabelled images without any extra knowledge. We propose an exceedingly simple method of conditioning an existing representation on a diverse data distribution and observe that a model trained on diverse examples acts as a better initialization. We extensively study our findings for the task of surface normal estimation and semantic segmentation from a single image. We improve surface normal estimation on NYU-v2 depth dataset and semantic segmentation on PASCAL VOC by 4% over base model. We did not use any task-specific knowledge or auxiliary tasks, neither changed hyper-parameters nor made any modification in the underlying neural network architecture.