Boosting Supervised Learning Performance with Co-training

TL;DR

This paper introduces a lightweight multi-task co-training framework that integrates self-supervised tasks into supervised learning, significantly improving perception model accuracy and domain adaptation with minimal additional computational cost.

Contribution

The paper presents a novel, flexible co-training framework that combines self-supervised tasks with supervised learning to enhance performance and domain adaptation efficiently.

Findings

Self-supervised tasks improve supervised model accuracy.

Framework demonstrates strong domain adaptation capabilities.

Minimal additional computation required.

Abstract

Deep learning perception models require a massive amount of labeled training data to achieve good performance. While unlabeled data is easy to acquire, the cost of labeling is prohibitive and could create a tremendous burden on companies or individuals. Recently, self-supervision has emerged as an alternative to leveraging unlabeled data. In this paper, we propose a new light-weight self-supervised learning framework that could boost supervised learning performance with minimum additional computation cost. Here, we introduce a simple and flexible multi-task co-training framework that integrates a self-supervised task into any supervised task. Our approach exploits pretext tasks to incur minimum compute and parameter overheads and minimal disruption to existing training pipelines. We demonstrate the effectiveness of our framework by using two self-supervised tasks, object detection and panoptic segmentation, on different perception models. Our results show that both self-supervised tasks can improve the accuracy of the supervised task and, at the same time, demonstrates strong domain adaption capability when used with additional unlabeled data.