Fine-tuning Vision Classifiers On A Budget

TL;DR

This paper introduces GTX, a method that leverages prior labeler accuracy estimates with a naive-Bayes model to improve label quality and quantity, enabling effective fine-tuning of vision models with fewer labels.

Contribution

The paper presents GTX, a novel approach that uses labeler accuracy estimates to enhance label quality, reducing labeling costs for fine-tuning vision classifiers.

Findings

GTX improves label efficiency in fine-tuning.

Using prior labeler accuracy estimates maintains model performance.

GTX reduces the need for extensive human labeling.

Abstract

Fine-tuning modern computer vision models requires accurately labeled data for which the ground truth may not exist, but a set of multiple labels can be obtained from labelers of variable accuracy. We tie the notion of label quality to confidence in labeler accuracy and show that, when prior estimates of labeler accuracy are available, using a simple naive-Bayes model to estimate the true labels allows us to label more data on a fixed budget without compromising label or fine-tuning quality. We present experiments on a dataset of industrial images that demonstrates that our method, called Ground Truth Extension (GTX), enables fine-tuning ML models using fewer human labels.