Comprehensive Design Space Exploration for Tensorized Neural Network Hardware Accelerators

TL;DR

This paper introduces a co-exploration framework that jointly optimizes tensor contraction paths, hardware architecture, and dataflow mapping to improve the deployment efficiency of tensorized neural networks on edge devices, achieving significant latency reductions.

Contribution

It presents a unified design space and a latency-driven search method for optimizing tensorized neural network deployment on hardware, addressing the gap between algorithmic and hardware-aware design.

Findings

Achieves up to 4x lower inference latency

Achieves up to 3.85x lower training latency

Demonstrates effectiveness on FPGA hardware

Abstract

High-order tensor decomposition has been widely adopted to obtain compact deep neural networks for edge deployment. However, existing studies focus primarily on its algorithmic advantages such as accuracy and compression ratio-while overlooking the hardware deployment efficiency. Such hardware-unaware designs often obscure the potential latency and energy benefits of tensorized models. Although several works attempt to reduce computational cost by optimizing the contraction sequence based on the number of multiply-accumulate operations, they typically neglect the underlying hardware characteristics, resulting in suboptimal real-world performance. We observe that the contraction path, hardware architecture, and dataflow mapping are tightly coupled and must be optimized jointly within a unified design space to maximize deployment efficiency on real devices. To this end, we propose a co-exploration framework that unifies these dimensions within a unified design space for efficient training and inference of tensorized neural networks on edge platforms. The framework formulates a latency oriented search objective and solves it via a global latency-driven exploration across the unified design space to achieve end-to-end model efficiency. The optimized configurations are implemented on a configurable FPGA kernel, achieving up to 4x and 3.85x lower inference and training latency compared with the dense baseline.