ONNX-to-Hardware Design Flow for Adaptive Neural-Network Inference on FPGAs

TL;DR

This paper presents an ONNX-to-hardware design flow enabling lightweight, adaptive neural network inference on FPGAs at the edge, emphasizing flexibility, energy efficiency, and support for evolving applications.

Contribution

It introduces an extension to a semi-integrated ONNX-to-hardware toolchain focused on approximate computing for adaptive neural network inference on FPGAs.

Findings

Supports energy-efficient, adaptive NN inference on FPGAs

Enables approximate computing with the ONNX-to-hardware flow

Facilitates flexible, long-term deployable neural network solutions

Abstract

The challenges involved in executing neural networks (NNs) at the edge include providing diversity, flexibility, and sustainability. That implies, for instance, supporting evolving applications and algorithms energy-efficiently. Using hardware or software accelerators can deliver fast and efficient computation of the NNs, while flexibility can be exploited to support long-term adaptivity. Nonetheless, handcrafting an NN for a specific device, despite the possibility of leading to an optimal solution, takes time and experience, and that's why frameworks for hardware accelerators are being developed. This work, starting from a preliminary semi-integrated ONNX-to-hardware toolchain [21], focuses on enabling approximate computing leveraging the distinctive ability of the original toolchain to favor adaptivity. The goal is to allow lightweight adaptable NN inference on FPGAs at the edge.