Are You Sure You Want To Do That? Classification with Verification

TL;DR

This paper introduces a classification model that uses verification with reference images to reduce memory requirements and improve sample efficiency, demonstrating competitive accuracy and emphasizing the importance of balancing recognition and verification.

Contribution

It proposes a novel verification-based classification approach with iterative queries, showing feasibility and advantages over traditional memorization-based methods.

Findings

Model matches baseline accuracy with fewer parameters

Verification reduces memory and improves sample efficiency

Balancing recognition and verification is crucial for optimal performance

Abstract

Classification systems typically act in isolation, meaning they are required to implicitly memorize the characteristics of all candidate classes in order to classify. The cost of this is increased memory usage and poor sample efficiency. We propose a model which instead verifies using reference images during the classification process, reducing the burden of memorization. The model uses iterative nondifferentiable queries in order to classify an image. We demonstrate that such a model is feasible to train and can match baseline accuracy while being more parameter efficient. However, we show that finding the correct balance between image recognition and verification is essential to pushing the model towards desired behavior, suggesting that a pipeline of recognition followed by verification is a more promising approach.