94% on CIFAR-10 in 3.29 Seconds on a Single GPU

TL;DR

This paper presents a highly efficient training method for CIFAR-10 that achieves near state-of-the-art accuracy within seconds on a single GPU, significantly speeding up research workflows.

Contribution

The paper introduces a novel training approach and a derandomized augmentation technique that drastically reduce training time for CIFAR-10.

Findings

Achieves 94% accuracy in 3.29 seconds on a single GPU

Proposes a derandomized horizontal flipping augmentation that outperforms standard methods

Demonstrates significant speedup in CIFAR-10 training without sacrificing accuracy

Abstract

CIFAR-10 is among the most widely used datasets in machine learning, facilitating thousands of research projects per year. To accelerate research and reduce the cost of experiments, we introduce training methods for CIFAR-10 which reach 94% accuracy in 3.29 seconds, 95% in 10.4 seconds, and 96% in 46.3 seconds, when run on a single NVIDIA A100 GPU. As one factor contributing to these training speeds, we propose a derandomized variant of horizontal flipping augmentation, which we show improves over the standard method in every case where flipping is beneficial over no flipping at all. Our code is released at https://github.com/KellerJordan/cifar10-airbench.