Can Self-Supervised Representation Learning Methods Withstand Distribution Shifts and Corruptions?

TL;DR

This paper evaluates how well self-supervised learning methods in computer vision withstand distribution shifts and image corruptions, revealing significant performance degradation under severe conditions and emphasizing the need for robustness-focused strategies.

Contribution

It provides a comprehensive empirical analysis of the robustness of various self-supervised learning approaches against distribution shifts and corruptions, highlighting their vulnerabilities.

Findings

Higher distribution shifts and corruptions reduce robustness significantly.

Self-supervised methods are less resilient under severe corruptions.

The study advocates for robustness-focused future research in self-supervised learning.

Abstract

Self-supervised learning in computer vision aims to leverage the inherent structure and relationships within data to learn meaningful representations without explicit human annotation, enabling a holistic understanding of visual scenes. Robustness in vision machine learning ensures reliable and consistent performance, enhancing generalization, adaptability, and resistance to noise, variations, and adversarial attacks. Self-supervised paradigms, namely contrastive learning, knowledge distillation, mutual information maximization, and clustering, have been considered to have shown advances in invariant learning representations. This work investigates the robustness of learned representations of self-supervised learning approaches focusing on distribution shifts and image corruptions in computer vision. Detailed experiments have been conducted to study the robustness of self-supervised learning methods on distribution shifts and image corruptions. The empirical analysis demonstrates a clear relationship between the performance of learned representations within self-supervised paradigms and the severity of distribution shifts and corruptions. Notably, higher levels of shifts and corruptions are found to significantly diminish the robustness of the learned representations. These findings highlight the critical impact of distribution shifts and image corruptions on the performance and resilience of self-supervised learning methods, emphasizing the need for effective strategies to mitigate their adverse effects. The study strongly advocates for future research in the field of self-supervised representation learning to prioritize the key aspects of safety and robustness in order to ensure practical applicability. The source code and results are available on GitHub.