True 4-Bit Quantized Convolutional Neural Network Training on CPU: Achieving Full-Precision Parity

TL;DR

This paper introduces a practical method for training 4-bit quantized convolutional neural networks on standard CPUs, achieving accuracy comparable to full-precision models while significantly reducing memory usage.

Contribution

The authors present a novel 4-bit training technique using only CPU operations, matching full-precision accuracy without specialized hardware or post-training quantization.

Findings

Achieves 92.34% accuracy on CIFAR-10 with 4-bit training on CPU

Maintains full-precision parity accuracy without GPU or specialized kernels

Demonstrates generalization to CIFAR-100 and rapid convergence on mobile devices

Abstract

Low-precision neural network training has emerged as a promising direction for reducing computational costs and democratizing access to deep learning research. However, existing 4-bit quantization methods either rely on expensive GPU infrastructure or suffer from significant accuracy degradation. In this work, we present a practical method for training convolutional neural networks at true 4-bit precision using standard PyTorch operations on commodity CPUs. We introduce a novel tanh-based soft weight clipping technique that, combined with symmetric quantization, dynamic per-layer scaling, and straight-through estimators, achieves stable convergence and competitive accuracy. Training a VGG-style architecture with 3.25 million parameters from scratch on CIFAR-10, our method achieves 92.34% test accuracy on Google Colab's free CPU tier -- matching full-precision baseline performance (92.5%) with only a 0.16% gap. We further validate on CIFAR-100, achieving 70.94% test accuracy across 100 classes with the same architecture and training procedure, demonstrating that 4-bit training from scratch generalizes to harder classification tasks. Both experiments achieve 8x memory compression over FP32 while maintaining exactly 15 unique weight values per layer throughout training. We additionally validate hardware independence by demonstrating rapid convergence on a consumer mobile device (OnePlus 9R), achieving 83.16% accuracy in only 6 epochs. To the best of our knowledge, no prior work has demonstrated 4-bit quantization-aware training achieving full-precision parity on standard CPU hardware without specialized kernels or post-training quantization.