Flexible Vector Integration in Embedded RISC-V SoCs for End to End CNN Inference Acceleration

TL;DR

This paper presents a flexible vector integration approach in embedded RISC-V SoCs that accelerates CNN inference by optimizing hardware placement, reducing bottlenecks, and leveraging RISC-V Vector 1.0 for balanced, power-efficient execution.

Contribution

It introduces a novel system integration and compilation model utilizing RISC-V Vector 1.0 to enhance CNN inference performance on resource-constrained embedded platforms.

Findings

Up to 9x speedup in image pre-processing.

Up to 3x reduction in YOLOv3 fallback layer execution time.

Demonstrates power-efficient, balanced computation on heterogeneous SoCs.

Abstract

The emergence of heterogeneity and domain-specific architectures targeting deep learning inference show great potential for enabling the deployment of modern CNNs on resource-constrained embedded platforms. A significant development is the diversification of custom hardware solely targeting the most expensive parts of CNNs. DLAs (deep learning accelerators) and NPUs (neural processing units), among others, can overcome the approaching limits of traditional silicon scaling and provide a solution to the power/performance tradeoff within embedded SoCs. Efficient DSA utilization requires proper system integration and a compilation/execution model for balanced execution in these heterogeneous architectures. There is a critical need for proper system integration and an efficient compilation/execution model for balanced execution in these heterogeneous architectures. This work highlights the hardware integration challenges for efficiently placing these units within the memory hierarchy and correct proximity to other execution blocks. We experimentally verify performance bottlenecks in CNN execution and pre/post-processing at runtime, where previous attention has generally been given to accelerator speedup alone. This work takes advantage of the ratification of the RISC-V Vector 1.0 extension and demonstrates its potential as a flexible target within a well-suited cache hierarchy scheme to reduce pre-processing bottlenecks and CPU fallback processes. Our results show up to a 9x speedup of image pre-processing and YOLOv3 fallback layer execution by up to 3x compared to CPU. We demonstrate RVV-1.0 in exposing a flexible programming model that can enable a balanced computation and memory footprint on accelerator-rich embedded SoCs supporting modern deep-learning dataflows while consuming less power than traditional parallel execution platforms.