A Cookbook of Self-Supervised Learning

TL;DR

This paper presents a comprehensive 'cookbook' of self-supervised learning, aiming to simplify and guide researchers through the complex process of training SSL models by providing foundational knowledge and practical recipes.

Contribution

It offers a structured collection of SSL methods and best practices, making the field more accessible and easier to experiment with for researchers and practitioners.

Findings

Provides a systematic overview of SSL components

Includes practical recipes for training SSL models

Aims to lower the barrier to entry in SSL research

Abstract

Self-supervised learning, dubbed the dark matter of intelligence, is a promising path to advance machine learning. Yet, much like cooking, training SSL methods is a delicate art with a high barrier to entry. While many components are familiar, successfully training a SSL method involves a dizzying set of choices from the pretext tasks to training hyper-parameters. Our goal is to lower the barrier to entry into SSL research by laying the foundations and latest SSL recipes in the style of a cookbook. We hope to empower the curious researcher to navigate the terrain of methods, understand the role of the various knobs, and gain the know-how required to explore how delicious SSL can be.