Parametric Instance Classification for Unsupervised Visual Feature Learning

TL;DR

This paper introduces Parametric Instance Classification (PIC), a simple and effective unsupervised visual feature learning method that simplifies the framework by using a parametric approach, matching state-of-the-art performance while improving efficiency.

Contribution

The paper proposes a novel parametric classification framework for unsupervised learning, along with techniques to enhance its effectiveness and practicality, such as a sliding-window data scheduler and negative sampling correction.

Findings

PIC achieves comparable performance to SimCLR and MoCo v2.

The sliding-window data scheduler improves instance visiting frequency.

Negative sampling correction reduces training time and memory usage.

Abstract

This paper presents parametric instance classification (PIC) for unsupervised visual feature learning. Unlike the state-of-the-art approaches which do instance discrimination in a dual-branch non-parametric fashion, PIC directly performs a one-branch parametric instance classification, revealing a simple framework similar to supervised classification and without the need to address the information leakage issue. We show that the simple PIC framework can be as effective as the state-of-the-art approaches, i.e. SimCLR and MoCo v2, by adapting several common component settings used in the state-of-the-art approaches. We also propose two novel techniques to further improve effectiveness and practicality of PIC: 1) a sliding-window data scheduler, instead of the previous epoch-based data scheduler, which addresses the extremely infrequent instance visiting issue in PIC and improves the effectiveness; 2) a negative sampling and weight update correction approach to reduce the training time and GPU memory consumption, which also enables application of PIC to almost unlimited training images. We hope that the PIC framework can serve as a simple baseline to facilitate future study.