Efficient ResNets: Residual Network Design

TL;DR

This paper presents a compact ResNet model with under 5 million parameters that achieves high accuracy on CIFAR-10, demonstrating efficient design for resource-constrained environments.

Contribution

The authors designed and trained a smaller ResNet model that outperforms larger models like ResNet18 on CIFAR-10, emphasizing efficiency and accuracy.

Findings

Achieved 96.04% test accuracy on CIFAR-10.

Model size is under 5 million parameters.

Outperforms ResNet18 despite fewer parameters.

Abstract

ResNets (or Residual Networks) are one of the most commonly used models for image classification tasks. In this project, we design and train a modified ResNet model for CIFAR-10 image classification. In particular, we aimed at maximizing the test accuracy on the CIFAR-10 benchmark while keeping the size of our ResNet model under the specified fixed budget of 5 million trainable parameters. Model size, typically measured as the number of trainable parameters, is important when models need to be stored on devices with limited storage capacity (e.g. IoT/edge devices). In this article, we present our residual network design which has less than 5 million parameters. We show that our ResNet achieves a test accuracy of 96.04% on CIFAR-10 which is much higher than ResNet18 (which has greater than 11 million trainable parameters) when equipped with a number of training strategies and suitable ResNet hyperparameters. Models and code are available at https://github.com/Nikunj-Gupta/Efficient_ResNets.