Fast Generation of Custom Floating-Point Spatial Filters on FPGAs

TL;DR

This paper introduces FPGA-based implementations of linear and nonlinear spatial filters using custom floating-point arithmetic, enabling real-time 1080p video processing and rapid algorithm prototyping for non-experts.

Contribution

It presents novel FPGA designs for spatial filters with custom floating-point, improving speed and flexibility for real-time video processing and simplifying development.

Findings

Processing 1080p video at 60 fps on low-cost FPGA

Custom floating-point reduces hardware complexity

Domain-specific language enables rapid prototyping

Abstract

Convolutional Neural Networks (CNNs) have been utilised in many image and video processing applications. The convolution operator, also known as a spatial filter, is usually a linear operation, but this linearity compromises essential features and details inherent in the non-linearity present in many applications. However, due to its slow processing, the use of a nonlinear spatial filter is a significant bottleneck in many software applications. Further, due to their complexity, they are difficult to accelerate in FPGA or VLSI architectures. This paper presents novel FPGA implementations of linear and nonlinear spatial filters. More specifically, the arithmetic computations are carried out in custom floating-point, enabling a tradeoff of precision and hardware compactness, reducing algorithm development time. Further, we show that it is possible to process video at a resolution of 1080p with a frame rate of 60 frames per second, using a low-cost FPGA board. Finally, we show that using a domain-specific language will allow the rapid prototyping of image processing algorithms in custom floating-point arithmetic, allowing non-experts to quickly develop real-time video processing applications.