Lets keep it simple, Using simple architectures to outperform deeper and more complex architectures
Seyyed Hossein Hasanpour, Mohammad Rouhani, Mohsen Fayyaz, Mohammad, Sabokrou

TL;DR
This paper introduces SimpleNet, a straightforward 13-layer CNN architecture that achieves competitive or superior accuracy compared to deeper, more complex models while significantly reducing computational and memory requirements, making it suitable for resource-limited systems.
Contribution
The paper presents a simple, principled design for CNNs that outperforms complex architectures like VGGNet and ResNet on multiple benchmarks with fewer parameters.
Findings
SimpleNet outperforms VGGNet, ResNet, and GoogleNet on several benchmarks.
Achieves state-of-the-art on CIFAR10 with fewer parameters.
Performs competitively on ImageNet with a simpler architecture.
Abstract
Major winning Convolutional Neural Networks (CNNs), such as AlexNet, VGGNet, ResNet, GoogleNet, include tens to hundreds of millions of parameters, which impose considerable computation and memory overhead. This limits their practical use for training, optimization and memory efficiency. On the contrary, light-weight architectures, being proposed to address this issue, mainly suffer from low accuracy. These inefficiencies mostly stem from following an ad hoc procedure. We propose a simple architecture, called SimpleNet, based on a set of designing principles, with which we empirically show, a well-crafted yet simple and reasonably deep architecture can perform on par with deeper and more complex architectures. SimpleNet provides a good tradeoff between the computation/memory efficiency and the accuracy. Our simple 13-layer architecture outperforms most of the deeper and complex…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Neural Network Applications · Adversarial Robustness in Machine Learning · Anomaly Detection Techniques and Applications
MethodsStochastic Gradient Descent · Weight Decay · SimpleNet · 1x1 Convolution · Batch Normalization · Convolution · *Communicated@Fast*How Do I Communicate to Expedia? · Max Pooling · Softmax
