Empirical Ablation and Ensemble Optimization of a Convolutional Neural Network for CIFAR-10 Classification

TL;DR

This paper empirically studies CNN modifications for CIFAR-10, showing that careful ablation and ensemble methods significantly improve accuracy over baseline models.

Contribution

It systematically evaluates 17 architectural and training modifications, demonstrating the importance of empirical selection and ensemble techniques for small-image classification.

Findings

Training duration extension improves accuracy steadily.

Structural redesigns can reduce accuracy despite increased complexity.

Ensemble of top configurations achieves up to 89.23% accuracy.

Abstract

Convolutional neural networks (CNNs) remain a central approach in image classification, but their performance depends strongly on architectural and training choices. This paper presents an empirical ablation-based study of CNN optimization for the CIFAR-10 benchmark. The study evaluates 17 progressive modifications involving training duration, learning-rate scheduling, dropout configuration, pooling strategy, network depth, filter arrangement, and dense-layer design. The goal is to identify which changes improve generalization and which increase complexity without improving performance. The baseline model achieved 79.5\% test accuracy. Extending training duration improved performance steadily, whereas several structural redesigns reduced accuracy despite greater architectural variation. Based on the strongest individual configurations, a weighted ensemble was constructed, achieving 86.38\% accuracy in the reduced-data setting and 89.23\% when trained using the full CIFAR-10 dataset. These results suggest that performance gains in CNN-based classification depend less on indiscriminate increases in depth or parameter count than on careful empirical selection of training and architectural modifications. The study therefore highlights the practical value of ablation-oriented optimization and ensemble learning for small-image classification.