Quantization Effects of Artificial Neural Networks for Embedded Edge-Computing Applications

TL;DR

This paper explores quantization techniques for neural networks in embedded scientific applications, demonstrating significant memory savings and ultra-low latency inference with novel hardware-constrained training methods.

Contribution

It introduces a genetic algorithm-based training approach for non-differentiable BNNs and evaluates quantization trade-offs in resource-constrained scenarios.

Findings

PTQ reduces memory four-fold while maintaining accuracy

Proposed GAs enable training of non-differentiable BNNs

Achieves inference latency of 10-15 ns without specialized hardware

Abstract

This paper examines the use of Quantized Neural Networks (QNNs) for two resource-constrained scientific applications: automated calibration of semi-conductor quantum bits (qubits) and scientific particle detectors. We evaluate the trade-offs between Post-Training Quantization (PTQ), Quantization-Aware Training (QAT), and ultra-low-bit Binary Neural Networks (BNNs) with respect to latency and resource usage. Our results demonstrate that PTQ achieves a four-fold reduction in memory usage for U-shaped CNN (U-Net) architectures while maintaining or slightly enhancing segmentation accuracy (e.g. from 89% to 90% for a small U-Net with 447 parameters). For the training of non-differentiable custom BNNs , we propose a novel, hardware-constrained learning approach using Genetic Algorithms (GAs). We showcase a LUT-based BNN architecture suitable for direct conversion to VHDL via the HCL4BNN framework. This method achieves nanosecond-scale inference latencies (10-15 ns) without requiring specialized DSP or BRAM resources.