Energy Efficient Video Fusion with Heterogeneous CPU-FPGA Devices

TL;DR

This paper develops an energy-efficient video fusion system using CPU and FPGA, demonstrating that workload-dependent adaptive selection between hardware accelerators optimizes energy and performance.

Contribution

It introduces an adaptive system that intelligently chooses between CPU SIMD and FPGA for wavelet transforms to maximize energy efficiency and performance.

Findings

FPGA accelerates video fusion by a factor of 2 in time and energy.

SIMD engine is preferable for smaller frame sizes.

Workload-dependent selection improves overall efficiency.

Abstract

This paper presents a complete video fusion system with hardware acceleration and investigates the energy trade-offs between computing in the CPU or the FPGA device. The video fusion application is based on the Dual-Tree Complex Wavelet Transforms (DT-CWT). In this work the transforms are mapped to a hardware accelerator using high-level synthesis tools for the FPGA and also vectorized code for the single instruction multiple data (SIMD) engine available in the CPU. The accelerated system reduces computation time and energy by a factor of 2. Moreover, the results show a key finding that the FPGA is not always the best choice for acceleration, and the SIMD engine should be selected when the wavelet decomposition reduces the frame size below a certain threshold. This dependency on workload size means that an adaptive system that intelligently selects between the SIMD engine and the FPGA achieves the most energy and performance efficiency point.