Self-Supervised Contextual Bandits in Computer Vision

TL;DR

This paper introduces a novel method combining self-supervised learning with contextual bandits to improve reward optimization in computer vision tasks, demonstrating significant gains across multiple datasets.

Contribution

It proposes a new approach that integrates self-supervision into contextual bandit algorithms, addressing the lack of implicit labels in early learning stages.

Findings

Substantial improvements in cumulative reward on eight datasets

Identification of cases where the method underperforms and alternative solutions

Enhanced data representation learning for better decision-making

Abstract

Contextual bandits are a common problem faced by machine learning practitioners in domains as diverse as hypothesis testing to product recommendations. There have been a lot of approaches in exploiting rich data representations for contextual bandit problems with varying degree of success. Self-supervised learning is a promising approach to find rich data representations without explicit labels. In a typical self-supervised learning scheme, the primary task is defined by the problem objective (e.g. clustering, classification, embedding generation etc.) and the secondary task is defined by the self-supervision objective (e.g. rotation prediction, words in neighborhood, colorization, etc.). In the usual self-supervision, we learn implicit labels from the training data for a secondary task. However, in the contextual bandit setting, we don't have the advantage of getting implicit labels due to lack of data in the initial phase of learning. We provide a novel approach to tackle this issue by combining a contextual bandit objective with a self supervision objective. By augmenting contextual bandit learning with self-supervision we get a better cumulative reward. Our results on eight popular computer vision datasets show substantial gains in cumulative reward. We provide cases where the proposed scheme doesn't perform optimally and give alternative methods for better learning in these cases.