Algorithm and Hardware Co-Design for Efficient Complex-Valued Uncertainty Estimation

TL;DR

This paper introduces Bayesian complex-valued neural networks with automated configuration search and FPGA-based accelerators, enabling uncertainty quantification and efficient hardware deployment for complex-valued tasks.

Contribution

It presents the first dropout-based Bayesian CVNNs, an automated search for optimal configurations, and FPGA accelerators, advancing uncertainty estimation and hardware efficiency in complex-valued neural networks.

Findings

Automated search effectively finds optimal configurations.

Accelerators achieve 4.5x to 13x speedups over GPU.

Models outperform existing methods in hardware efficiency.

Abstract

Complex-Valued Neural Networks (CVNNs) have significant advantages in handling tasks that involve complex numbers. However, existing CVNNs are unable to quantify predictive uncertainty. We propose, for the first time, dropout-based Bayesian Complex-Valued Neural Networks (BayesCVNNs) to enable uncertainty quantification for complex-valued applications, exhibiting broad applicability and efficiency for hardware implementation due to modularity. Furthermore, as the dual-part nature of complex values significantly broadens the design space and enables novel configurations based on layer-mixing and part-mixing, we introduce an automated search approach to effectively identify optimal configurations for both real and imaginary components. To facilitate deployment, we present a framework that generates customized FPGA-based accelerators for BayesCVNNs, leveraging a set of optimized building blocks. Experiments demonstrate the best configuration can be effectively found via the automated search, attaining higher performance with lower hardware costs compared with manually crafted models. The optimized accelerators achieve approximately 4.5x and 13x speedups on different models with less than 10% power consumption compared to GPU implementations, and outperform existing work in both algorithm and hardware aspects. Our code is publicly available at: https://github.com/zehuanzhang/BayesCVNN.git.