STAR: Noisy Semi-Supervised Transfer Learning for Visual Classification

TL;DR

This paper introduces a noisy semi-supervised transfer learning framework that efficiently leverages unlabeled data for visual classification, significantly improving accuracy and robustness while reducing computational resources needed.

Contribution

The paper presents a novel framework combining transfer learning and noisy self-training tailored for small-scale unlabeled data scenarios in visual classification.

Findings

Significant accuracy improvements over state-of-the-art methods.

Reduces compute time by 6x and memory by 5x.

Enhances robustness without explicit adversarial training.

Abstract

Semi-supervised learning (SSL) has proven to be effective at leveraging large-scale unlabeled data to mitigate the dependency on labeled data in order to learn better models for visual recognition and classification tasks. However, recent SSL methods rely on unlabeled image data at a scale of billions to work well. This becomes infeasible for tasks with relatively fewer unlabeled data in terms of runtime, memory and data acquisition. To address this issue, we propose noisy semi-supervised transfer learning, an efficient SSL approach that integrates transfer learning and self-training with noisy student into a single framework, which is tailored for tasks that can leverage unlabeled image data on a scale of thousands. We evaluate our method on both binary and multi-class classification tasks, where the objective is to identify whether an image displays people practicing sports or the type of sport, as well as to identify the pose from a pool of popular yoga poses. Extensive experiments and ablation studies demonstrate that by leveraging unlabeled data, our proposed framework significantly improves visual classification, especially in multi-class classification settings compared to state-of-the-art methods. Moreover, incorporating transfer learning not only improves classification performance, but also requires 6x less compute time and 5x less memory. We also show that our method boosts robustness of visual classification models, even without specifically optimizing for adversarial robustness.