Automated and Holistic Co-design of Neural Networks and ASICs for Enabling In-Pixel Intelligence

TL;DR

This paper introduces a multi-objective Bayesian optimization framework that jointly co-designs neural networks and ASIC hardware for ultra-constrained edge-AI systems, enabling real-time in-pixel intelligence with optimal trade-offs.

Contribution

It presents a novel integrated search method combining neural network and ASIC design, overcoming limitations of theoretical metrics and manual approaches in complex, constrained design spaces.

Findings

Identified Pareto-optimal neural network and ASIC configurations.

Demonstrated real-time feature extraction within pixel-level ASICs.

Showed improved efficiency and accuracy over traditional methods.

Abstract

Extreme edge-AI systems, such as those in readout ASICs for radiation detection, must operate under stringent hardware constraints such as micron-level dimensions, sub-milliwatt power, and nanosecond-scale speed while providing clear accuracy advantages over traditional architectures. Finding ideal solutions means identifying optimal AI and ASIC design choices from a design space that has explosively expanded during the merger of these domains, creating non-trivial couplings which together act upon a small set of solutions as constraints tighten. It is impractical, if not impossible, to manually determine ideal choices among possibilities that easily exceed billions even in small-size problems. Existing methods to bridge this gap have leveraged theoretical understanding of hardware to f architecture search. However, the assumptions made in computing such theoretical metrics are too idealized to provide sufficient guidance during the difficult search for a practical implementation. Meanwhile, theoretical estimates for many other crucial metrics (like delay) do not even exist and are similarly variable, dependent on parameters of the process design kit (PDK). To address these challenges, we present a study that employs intelligent search using multi-objective Bayesian optimization, integrating both neural network search and ASIC synthesis in the loop. This approach provides reliable feedback on the collective impact of all cross-domain design choices. We showcase the effectiveness of our approach by finding several Pareto-optimal design choices for effective and efficient neural networks that perform real-time feature extraction from input pulses within the individual pixels of a readout ASIC.