A general approach to bridge the reality-gap

TL;DR

This paper introduces an unsupervised deep learning transformation that aligns arbitrary images with a canonical distribution, improving the transferability of models to real-world data without extensive real-world training data.

Contribution

It proposes a general, unsupervised transformation method to bridge the reality gap, reducing reliance on domain-specific adaptation and transfer learning.

Findings

Recovered half of the performance loss on distorted datasets

Effective on real-world images captured under varying lighting conditions

Applicable to pre-trained ImageNet models

Abstract

Employing machine learning models in the real world requires collecting large amounts of data, which is both time consuming and costly to collect. A common approach to circumvent this is to leverage existing, similar data-sets with large amounts of labelled data. However, models trained on these canonical distributions do not readily transfer to real-world ones. Domain adaptation and transfer learning are often used to breach this "reality gap", though both require a substantial amount of real-world data. In this paper we discuss a more general approach: we propose learning a general transformation to bring arbitrary images towards a canonical distribution where we can naively apply the trained machine learning models. This transformation is trained in an unsupervised regime, leveraging data augmentation to generate off-canonical examples of images and training a Deep Learning model to recover their original counterpart. We quantify the performance of this transformation using pre-trained ImageNet classifiers, demonstrating that this procedure can recover half of the loss in performance on the distorted data-set. We then validate the effectiveness of this approach on a series of pre-trained ImageNet models on a real world data set collected by printing and photographing images in different lighting conditions.