Self-Paced Learning with Adaptive Deep Visual Embeddings

TL;DR

This paper introduces SPL-ADVisE, a novel training protocol combining self-paced learning and deep metric learning to improve convergence and accuracy in deep neural network training for image classification.

Contribution

It presents an end-to-end method that uses learned visual embeddings to dynamically select training samples, enhancing training efficiency and performance.

Findings

Faster convergence across multiple datasets.

Higher final accuracy than other SPL methods.

Effective on both coarse and fine-grained visual recognition tasks.

Abstract

Selecting the most appropriate data examples to present a deep neural network (DNN) at different stages of training is an unsolved challenge. Though practitioners typically ignore this problem, a non-trivial data scheduling method may result in a significant improvement in both convergence and generalization performance. In this paper, we introduce Self-Paced Learning with Adaptive Deep Visual Embeddings (SPL-ADVisE), a novel end-to-end training protocol that unites self-paced learning (SPL) and deep metric learning (DML). We leverage the Magnet Loss to train an embedding convolutional neural network (CNN) to learn a salient representation space. The student CNN classifier dynamically selects similar instance-level training examples to form a mini-batch, where the easiness from the cross-entropy loss and the true diverseness of examples from the learned metric space serve as sample importance priors. To demonstrate the effectiveness of SPL-ADVisE, we use deep CNN architectures for the task of supervised image classification on several coarse- and fine-grained visual recognition datasets. Results show that, across all datasets, the proposed method converges faster and reaches a higher final accuracy than other SPL variants, particularly on fine-grained classes.