Consistent Relative Confidence and Label-Free Model Selection for Convolutional Neural Networks

TL;DR

This paper introduces a label-free CNN model selection method based on consistent relative confidence, enabling effective model choice without labeled data, which is useful when labeling is costly or delayed.

Contribution

The paper presents a novel approach for CNN model selection using only unlabeled data, bypassing the need for traditional performance metrics requiring labels.

Findings

Effective model selection on benchmark datasets

No labeled data needed for model comparison

Demonstrates efficiency and practical utility

Abstract

In this paper, we are concerned with image classification with deep convolutional neural networks (CNNs). We focus on the following question: given a set of candidate CNN models, how to select the right one with the best generalization property for the current task? Current model selection methods all require access to a batch of labeled data for computing a pre-specified performance metric, such as the cross-entropy loss, the classification error rate and the negative log-likelihood. In many practical cases, labels are not available in time as labeling itself is a time-consuming and expensive task. To this end, we propose an approach to CNN model selection using only unlabeled data. We develop this method based on a principle termed consistent relative confidence. Experimental results on benchmark datasets demonstrate the effectiveness and efficiency of our method.