# Semi-supervised Learning with Contrastive Predicative Coding

## TL;DR

This paper introduces contrastive predictive coding-based semi-supervised learning models that enhance discriminative capabilities and scalability for large datasets, addressing limitations of previous SSL methods.

## Contribution

It proposes two scalable, inductive SSL models, cpc-SSL and ccpc-SSL, leveraging contrastive predictive coding to improve performance with limited labels.

## Key findings

- Models effectively exploit unlabeled data through shared information extraction.
- Approaches scale well to large datasets like ImageNet.
- Demonstrated improved discriminative power in semi-supervised settings.

## Abstract

Semi-supervised learning (SSL) provides a powerful framework for leveraging unlabeled data when labels are limited or expensive to obtain. SSL algorithms based on deep neural networks have recently proven successful on standard benchmark tasks. However, many of them have thus far been either inflexible, inefficient or non-scalable. This paper explores recently developed contrastive predictive coding technique to improve discriminative power of deep learning models when a large portion of labels are absent. Two models, cpc-SSL and a class conditional variant~(ccpc-SSL) are presented. They effectively exploit the unlabeled data by extracting shared information between different parts of the (high-dimensional) data. The proposed approaches are inductive, and scale well to very large datasets like ImageNet, making them good candidates in real-world large scale applications.