Improving image classifiers for small datasets by learning rate adaptations

TL;DR

This paper presents a method that dynamically adjusts learning rates to significantly improve the accuracy and training speed of image classifiers, especially on small datasets, validated on CIFAR-10 and diagnostic images.

Contribution

It introduces a technique for learning rate adaptation that enhances classifier performance and training efficiency on small datasets, combining established methods effectively.

Findings

2-10x speedup in training time

Near state-of-the-art accuracy achieved

Effective on small and unbalanced datasets

Abstract

Our paper introduces an efficient combination of established techniques to improve classifier performance, in terms of accuracy and training time. We achieve two-fold to ten-fold speedup in nearing state of the art accuracy, over different model architectures, by dynamically tuning the learning rate. We find it especially beneficial in the case of a small dataset, where reliability of machine reasoning is lower. We validate our approach by comparing our method versus vanilla training on CIFAR-10. We also demonstrate its practical viability by implementing on an unbalanced corpus of diagnostic images.