Improving out-of-distribution generalization via multi-task self-supervised pretraining

TL;DR

This paper demonstrates that multi-task self-supervised pretraining improves out-of-distribution domain generalization in computer vision, outperforming supervised methods especially under large domain shifts.

Contribution

It introduces a new self-supervised pretext task involving Gabor filter responses and shows multi-task learning enhances domain generalization.

Findings

Self-supervised features outperform supervised ones under large domain shifts.

Multi-task learning with compatible pretext tasks improves generalization.

Self-supervised features show better object localization.

Abstract

Self-supervised feature representations have been shown to be useful for supervised classification, few-shot learning, and adversarial robustness. We show that features obtained using self-supervised learning are comparable to, or better than, supervised learning for domain generalization in computer vision. We introduce a new self-supervised pretext task of predicting responses to Gabor filter banks and demonstrate that multi-task learning of compatible pretext tasks improves domain generalization performance as compared to training individual tasks alone. Features learnt through self-supervision obtain better generalization to unseen domains when compared to their supervised counterpart when there is a larger domain shift between training and test distributions and even show better localization ability for objects of interest. Self-supervised feature representations can also be combined with other domain generalization methods to further boost performance.